[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:50:57 dist-test-slave-p0h9 krb5kdc[16199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:50:57 dist-test-slave-p0h9 krb5kdc[16199](info): set up 2 sockets
Jun 19 08:50:57 dist-test-slave-p0h9 krb5kdc[16199](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 08:50:58.797189 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:50:58 dist-test-slave-p0h9 krb5kdc[16199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:50:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:50:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:50:58.915493 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39855
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:39855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:50:58.997450 16212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:50:58.997709 16212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:50:58.997743 16212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:50:58.998893 16212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:50:58.998960 16212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:50:58.998986 16212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:50:58.999006 16212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:50:58.999038 16212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:50:59.000505 16212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:39855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:50:59.000873 16212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:50:59.001108 16212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:50:59.005993 16218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:50:59.005996 16220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:50:59.005996 16217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:50:59.006280 16212 server_base.cc:1034] running on GCE node
I20260619 08:50:59.006572 16212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:50:59.006876 16212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:50:59.008052 16212 hybrid_clock.cc:648] HybridClock initialized: now 1781859059008033 us; error 29 us; skew 500 ppm
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:50:59.010675 16212 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:50:59.011166 16212 webserver.cc:460] Webserver started at http://127.15.208.62:35009/ using document root <none> and password file <none>
I20260619 08:50:59.011394 16212 fs_manager.cc:359] Metadata directory not provided
I20260619 08:50:59.011462 16212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:50:59.011617 16212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:50:59.012678 16212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "d836f460127a435f9a7aee2a7c80c2ac"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "1903d97725203b476a63471e368e3018"
server_key_iv: "950d49756824606959e873a64dc842f3"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.013119 16212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "d836f460127a435f9a7aee2a7c80c2ac"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "1903d97725203b476a63471e368e3018"
server_key_iv: "950d49756824606959e873a64dc842f3"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.014434 16212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:50:59.015183 16227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.015359 16212 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:50:59.015463 16212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "d836f460127a435f9a7aee2a7c80c2ac"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "1903d97725203b476a63471e368e3018"
server_key_iv: "950d49756824606959e873a64dc842f3"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.015543 16212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:50:59.034345 16212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:50:59.038208 16212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:50:59.038398 16212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:50:59.044289 16212 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:39855
I20260619 08:50:59.044314 16279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:39855 every 8 connection(s)
I20260619 08:50:59.044771 16212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:50:59.045887 16280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:50:59.049675 16280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac: Bootstrap starting.
I20260619 08:50:59.050156 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 16212
I20260619 08:50:59.050261 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:50:59.050484 16280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac: Neither blocks nor log segments found. Creating new log.
I20260619 08:50:59.050761 16280 log.cc:824] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac: Log is configured to *not* fsync() on all Append() calls
I20260619 08:50:59.051065 16192 external_mini_cluster.cc:1428] Setting key 3329f35d0f0a116d40496d341ca41a32
I20260619 08:50:59.051610 16280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac: No bootstrap required, opened a new log
I20260619 08:50:59.054047 16280 raft_consensus.cc:348] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d836f460127a435f9a7aee2a7c80c2ac" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39855 } }
I20260619 08:50:59.054231 16280 raft_consensus.cc:374] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:50:59.054277 16280 raft_consensus.cc:729] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d836f460127a435f9a7aee2a7c80c2ac, State: Initialized, Role: FOLLOWER
I20260619 08:50:59.054430 16280 consensus_queue.cc:260] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d836f460127a435f9a7aee2a7c80c2ac" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39855 } }
I20260619 08:50:59.054528 16280 raft_consensus.cc:388] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:50:59.054576 16280 raft_consensus.cc:482] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:50:59.054629 16280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:50:59.055732 16280 raft_consensus.cc:504] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d836f460127a435f9a7aee2a7c80c2ac" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39855 } }
I20260619 08:50:59.055845 16280 leader_election.cc:302] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d836f460127a435f9a7aee2a7c80c2ac; no voters:
I20260619 08:50:59.056017 16280 leader_election.cc:288] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:50:59.056058 16285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:50:59.056205 16285 raft_consensus.cc:686] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [term 1 LEADER]: Becoming Leader. State: Replica: d836f460127a435f9a7aee2a7c80c2ac, State: Running, Role: LEADER
I20260619 08:50:59.056273 16280 sys_catalog.cc:564] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:50:59.056288 16285 consensus_queue.cc:237] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d836f460127a435f9a7aee2a7c80c2ac" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39855 } }
I20260619 08:50:59.056653 16285 sys_catalog.cc:455] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader d836f460127a435f9a7aee2a7c80c2ac. Latest consensus state: current_term: 1 leader_uuid: "d836f460127a435f9a7aee2a7c80c2ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d836f460127a435f9a7aee2a7c80c2ac" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39855 } } }
I20260619 08:50:59.056764 16285 sys_catalog.cc:458] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [sys.catalog]: This master's current role is: LEADER
I20260619 08:50:59.056908 16286 sys_catalog.cc:455] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d836f460127a435f9a7aee2a7c80c2ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d836f460127a435f9a7aee2a7c80c2ac" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39855 } } }
I20260619 08:50:59.056975 16286 sys_catalog.cc:458] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac [sys.catalog]: This master's current role is: LEADER
W20260619 08:50:59.058151 16301 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:50:59.058213 16301 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:50:59.060876 16291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:50:59.061081 16291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:50:59.062696 16291 catalog_manager.cc:1347] Generated new cluster ID: 6c26ff4e503b45488d50e716587c1d50
I20260619 08:50:59.062754 16291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:50:59.071676 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:50:59.052051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.15.208.62:39855)
0619 08:50:59.052218 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 08:50:59.052228 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:50:59.052785 (+ 557us) server_negotiation.cc:407] Connection header received
0619 08:50:59.055129 (+ 2344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:50:59.055139 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:50:59.055204 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:50:59.055351 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:50:59.058513 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.059864 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.061541 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.062155 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.062255 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:50:59.067237 (+ 4982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:50:59.067241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:50:59.067242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:50:59.067253 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:50:59.069444 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.069988 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.069991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.069992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.070126 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.070432 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.070433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.070433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.070727 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:50:59.070871 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 08:50:59.071308 (+ 437us) server_negotiation.cc:299] Negotiation successful
0619 08:50:59.071448 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260619 08:50:59.071941 16291 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:50:59.072746 16291 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:50:59.077807 16291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d836f460127a435f9a7aee2a7c80c2ac: Generated new TSK 0
I20260619 08:50:59.077991 16291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:50:59.105973 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39855
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:50:59.187003 16306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:50:59.187265 16306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:50:59.187304 16306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:50:59.188442 16306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:50:59.188520 16306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:50:59.188570 16306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:50:59.190066 16306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:50:59.190342 16306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:50:59.190584 16306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:50:59.195914 16312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:50:59.196030 16306 server_base.cc:1034] running on GCE node
W20260619 08:50:59.195935 16311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:50:59.195986 16314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:50:59.196370 16306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:50:59.196646 16306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:50:59.197806 16306 hybrid_clock.cc:648] HybridClock initialized: now 1781859059197791 us; error 30 us; skew 500 ppm
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:50:59.200176 16306 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:50:59.200541 16306 webserver.cc:460] Webserver started at http://127.15.208.1:35757/ using document root <none> and password file <none>
I20260619 08:50:59.200811 16306 fs_manager.cc:359] Metadata directory not provided
I20260619 08:50:59.200858 16306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:50:59.200954 16306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:50:59.202147 16306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "277d94d17e794cf3a57d58705fb53c9d"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "620de4d4c12b43f3e3f1c68c14c56bbd"
server_key_iv: "8121903936712a4d6ab47e367ec2a675"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.202601 16306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "277d94d17e794cf3a57d58705fb53c9d"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "620de4d4c12b43f3e3f1c68c14c56bbd"
server_key_iv: "8121903936712a4d6ab47e367ec2a675"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.203966 16306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:50:59.204833 16321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.205026 16306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:50:59.205109 16306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "277d94d17e794cf3a57d58705fb53c9d"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "620de4d4c12b43f3e3f1c68c14c56bbd"
server_key_iv: "8121903936712a4d6ab47e367ec2a675"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.205165 16306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:50:59.234205 16306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:50:59.236217 16306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:50:59.236341 16306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:50:59.236558 16306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:50:59.236938 16306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:50:59.236977 16306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.237031 16306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:50:59.237064 16306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.243956 16306 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:43905
I20260619 08:50:59.244026 16434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:43905 every 8 connection(s)
I20260619 08:50:59.244360 16306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:50:59.251309 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 16306
I20260619 08:50:59.251421 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:50:59.251592 16192 external_mini_cluster.cc:1428] Setting key 4827cefeeb0169d9c9dbeca63eef4197
I20260619 08:50:59.254354 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:50:59.245206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:41623 (local address 127.15.208.62:39855)
0619 08:50:59.245283 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:50:59.245285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:50:59.245541 (+ 256us) server_negotiation.cc:407] Connection header received
0619 08:50:59.246016 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:50:59.246017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:50:59.246028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:50:59.246084 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:50:59.247156 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.247667 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.248675 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.248886 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.248940 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:50:59.251141 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:50:59.251142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:50:59.251143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:50:59.251151 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:50:59.252970 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.253370 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.253372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.253372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.253429 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.253707 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.253708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.253708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.253824 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:50:59.253901 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:50:59.254179 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 08:50:59.254261 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:50:59.254566 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:50:59.245176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39855 (local address 127.15.208.1:41623)
0619 08:50:59.245381 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 08:50:59.245419 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:50:59.245917 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:50:59.246157 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:50:59.246160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:50:59.246825 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:50:59.247051 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:50:59.247057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.247773 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.247775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:50:59.248590 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:50:59.248593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.248956 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.248957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:50:59.249073 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:50:59.249543 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:50:59.249551 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:50:59.251023 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:50:59.253064 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:50:59.253071 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:50:59.253073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:50:59.253295 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:50:59.253518 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:50:59.253519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:50:59.253519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:50:59.253630 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:50:59.253892 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:50:59.253898 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:50:59.254041 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 08:50:59.254158 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 08:50:59.254320 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":119,"threads_started":1}
I20260619 08:50:59.255028 16435 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39855
I20260619 08:50:59.255146 16435 heartbeater.cc:464] Registering TS with master...
I20260619 08:50:59.255319 16435 heartbeater.cc:511] Master 127.15.208.62:39855 requested a full tablet report, sending...
I20260619 08:50:59.255924 16244 ts_manager.cc:194] Registered new tserver with Master: 277d94d17e794cf3a57d58705fb53c9d (127.15.208.1:43905)
I20260619 08:50:59.256597 16244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:41623
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:50:59.275957 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39855
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:50:59.357072 16440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:50:59.357273 16440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:50:59.357297 16440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:50:59.358548 16440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:50:59.358590 16440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:50:59.358616 16440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:50:59.360044 16440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:50:59.360281 16440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:50:59.360543 16440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:50:59.365417 16446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:50:59.365458 16445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:50:59.365429 16448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:50:59.365763 16440 server_base.cc:1034] running on GCE node
I20260619 08:50:59.365983 16440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:50:59.366214 16440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:50:59.367380 16440 hybrid_clock.cc:648] HybridClock initialized: now 1781859059367367 us; error 26 us; skew 500 ppm
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:50:59.369974 16440 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:50:59.370479 16440 webserver.cc:460] Webserver started at http://127.15.208.2:39835/ using document root <none> and password file <none>
I20260619 08:50:59.370723 16440 fs_manager.cc:359] Metadata directory not provided
I20260619 08:50:59.370790 16440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:50:59.370923 16440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:50:59.371989 16440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "5e1e4cd92aa04b37a95cae9a0d31cd3b"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "069e02fee3f2e24ddacac2d09b4e65e3"
server_key_iv: "7246d398afdac3280b040945f0dd8e58"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.372409 16440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "5e1e4cd92aa04b37a95cae9a0d31cd3b"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "069e02fee3f2e24ddacac2d09b4e65e3"
server_key_iv: "7246d398afdac3280b040945f0dd8e58"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.373780 16440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:50:59.374485 16455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.374634 16440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.374739 16440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "5e1e4cd92aa04b37a95cae9a0d31cd3b"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "069e02fee3f2e24ddacac2d09b4e65e3"
server_key_iv: "7246d398afdac3280b040945f0dd8e58"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.374816 16440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:50:59.399658 16440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:50:59.401706 16440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:50:59.401870 16440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:50:59.402105 16440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:50:59.402408 16440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:50:59.402469 16440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.402513 16440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:50:59.402542 16440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.408938 16440 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:36659
I20260619 08:50:59.409078 16569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:36659 every 8 connection(s)
I20260619 08:50:59.409329 16440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:50:59.411031 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 16440
I20260619 08:50:59.411109 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:50:59.411276 16192 external_mini_cluster.cc:1428] Setting key 2cb428d4c9d8c867f0e0e8fab1644fc9
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:50:59.419013 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:50:59.410035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:35551 (local address 127.15.208.62:39855)
0619 08:50:59.410080 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:50:59.410083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:50:59.410286 (+ 203us) server_negotiation.cc:407] Connection header received
0619 08:50:59.410810 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:50:59.410811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:50:59.410825 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:50:59.410874 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:50:59.412046 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.412523 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.413629 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.413839 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.413927 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:50:59.416136 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:50:59.416138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:50:59.416138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:50:59.416146 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:50:59.417674 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.418081 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.418082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.418083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.418147 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.418388 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.418389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.418389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.418501 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:50:59.418577 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:50:59.418842 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 08:50:59.418927 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:50:59.419216 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:50:59.410010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39855 (local address 127.15.208.2:35551)
0619 08:50:59.410144 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:50:59.410189 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:50:59.410696 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:50:59.410941 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:50:59.410945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:50:59.411725 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:50:59.411955 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:50:59.411963 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.412625 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.412626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:50:59.413529 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:50:59.413531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.413935 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.413936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:50:59.414080 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:50:59.414616 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:50:59.414624 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:50:59.416040 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:50:59.417771 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:50:59.417777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:50:59.417780 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:50:59.417996 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:50:59.418213 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:50:59.418213 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:50:59.418214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:50:59.418315 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:50:59.418579 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:50:59.418585 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:50:59.418713 (+ 128us) client_negotiation.cc:769] Sending connection context
0619 08:50:59.418811 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 08:50:59.418958 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260619 08:50:59.419797 16570 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39855
I20260619 08:50:59.419896 16570 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
I20260619 08:50:59.420101 16570 heartbeater.cc:511] Master 127.15.208.62:39855 requested a full tablet report, sending...
I20260619 08:50:59.420475 16244 ts_manager.cc:194] Registered new tserver with Master: 5e1e4cd92aa04b37a95cae9a0d31cd3b (127.15.208.2:36659)
I20260619 08:50:59.420936 16244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:35551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:50:59.437139 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39855
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:50:59.520660 16575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:50:59.520998 16575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:50:59.521025 16575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:50:59.522193 16575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:50:59.522234 16575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:50:59.522260 16575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:50:59.523739 16575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:50:59.523981 16575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:50:59.524219 16575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:50:59.529137 16580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:50:59.529249 16575 server_base.cc:1034] running on GCE node
W20260619 08:50:59.529100 16583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:50:59.529114 16581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:50:59.529536 16575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:50:59.529799 16575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:50:59.530992 16575 hybrid_clock.cc:648] HybridClock initialized: now 1781859059530978 us; error 31 us; skew 500 ppm
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:50:59.533618 16575 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:50:59.534099 16575 webserver.cc:460] Webserver started at http://127.15.208.3:40469/ using document root <none> and password file <none>
I20260619 08:50:59.534330 16575 fs_manager.cc:359] Metadata directory not provided
I20260619 08:50:59.534397 16575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:50:59.534545 16575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:50:59.535573 16575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "257b29eadbfc4aa79e7a2fa8566b3a2e"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "a301e051d0684376906bcefb50f13026"
server_key_iv: "b68efe178dc453ecfd86509246a682c2"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.535934 16575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "257b29eadbfc4aa79e7a2fa8566b3a2e"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "a301e051d0684376906bcefb50f13026"
server_key_iv: "b68efe178dc453ecfd86509246a682c2"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.537299 16575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:50:59.538005 16590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.538199 16575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:50:59.538297 16575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "257b29eadbfc4aa79e7a2fa8566b3a2e"
format_stamp: "Formatted at 2026-06-19 08:50:59 on dist-test-slave-p0h9"
server_key: "a301e051d0684376906bcefb50f13026"
server_key_iv: "b68efe178dc453ecfd86509246a682c2"
server_key_version: "encryptionkey@0"
I20260619 08:50:59.538373 16575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:50:59.556154 16575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:50:59.558811 16575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:50:59.558977 16575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:50:59.559206 16575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:50:59.559504 16575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:50:59.559566 16575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.559608 16575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:50:59.559633 16575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:50:59.565799 16575 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:37155
I20260619 08:50:59.565841 16705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:37155 every 8 connection(s)
I20260619 08:50:59.566154 16575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:50:59.571893 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 16575
I20260619 08:50:59.571979 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:50:59.572170 16192 external_mini_cluster.cc:1428] Setting key 892bca7bfa42695cba41e4d17adb1a0c
I20260619 08:50:59.575217 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:50:59.566949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:50583 (local address 127.15.208.62:39855)
0619 08:50:59.567013 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:50:59.567016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:50:59.567242 (+ 226us) server_negotiation.cc:407] Connection header received
0619 08:50:59.567698 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:50:59.567699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:50:59.567710 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:50:59.567761 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:50:59.568642 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.569194 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.570171 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.570347 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.570398 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:50:59.572646 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:50:59.572648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:50:59.572648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:50:59.572655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:50:59.574122 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.574473 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.574474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.574474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.574523 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.574717 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.574717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.574717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.574812 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:50:59.574861 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:50:59.575096 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 08:50:59.575155 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:50:59.575361 16708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:50:59.566929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39855 (local address 127.15.208.3:50583)
0619 08:50:59.567118 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 08:50:59.567156 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:50:59.567619 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:50:59.567814 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:50:59.567818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:50:59.568365 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:50:59.568564 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:50:59.568569 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.569270 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.569271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:50:59.570086 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:50:59.570087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.570416 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.570416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:50:59.570518 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:50:59.571069 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:50:59.571077 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:50:59.572543 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:50:59.574195 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:50:59.574201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:50:59.574202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:50:59.574410 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:50:59.574571 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:50:59.574572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:50:59.574572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:50:59.574666 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:50:59.574867 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:50:59.574871 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:50:59.574974 (+ 103us) client_negotiation.cc:769] Sending connection context
0619 08:50:59.575054 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:50:59.575167 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260619 08:50:59.575768 16706 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39855
I20260619 08:50:59.575843 16706 heartbeater.cc:464] Registering TS with master...
I20260619 08:50:59.575975 16706 heartbeater.cc:511] Master 127.15.208.62:39855 requested a full tablet report, sending...
I20260619 08:50:59.576310 16244 ts_manager.cc:194] Registered new tserver with Master: 257b29eadbfc4aa79e7a2fa8566b3a2e (127.15.208.3:37155)
I20260619 08:50:59.576674 16244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:50583
I20260619 08:50:59.584280 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:50:59.607512 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:50:59.599966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.15.208.1:43905)
0619 08:50:59.600130 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 08:50:59.600135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:50:59.600146 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:50:59.600171 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:50:59.600173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:50:59.600187 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:50:59.600258 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:50:59.600828 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.601598 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.602291 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.602633 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.602691 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:50:59.604870 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:50:59.604879 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:50:59.604879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:50:59.604892 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:50:59.606247 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.606634 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.606638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.606638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.606688 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.606888 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.606889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.606889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.607024 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:50:59.607175 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 08:50:59.607267 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 08:50:59.607362 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
W20260619 08:50:59.607885 16349 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54280
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:50:59.629637 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:50:59.622864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54284 (local address 127.15.208.1:43905)
0619 08:50:59.622932 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:50:59.622934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:50:59.623051 (+ 117us) server_negotiation.cc:407] Connection header received
0619 08:50:59.623115 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:50:59.623116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:50:59.623127 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:50:59.623168 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:50:59.623743 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.624232 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.625033 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:50:59.625232 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:50:59.625286 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:50:59.627504 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:50:59.627505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:50:59.627505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:50:59.627513 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:50:59.628694 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.629055 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.629056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.629056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.629105 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:50:59.629263 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:50:59.629264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:50:59.629264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:50:59.629350 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:50:59.629437 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:50:59.629510 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 08:50:59.629571 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:50:59.630203 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 16306
I20260619 08:50:59.634718 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 16440
I20260619 08:50:59.638906 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 16575
I20260619 08:50:59.643070 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 16212
2026-06-19T08:50:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1899 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16720](info): set up 2 sockets
Jun 19 08:50:59 dist-test-slave-p0h9 krb5kdc[16720](info): commencing operation
krb5kdc: starting...
W20260619 08:51:00.684705 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:00 dist-test-slave-p0h9 krb5kdc[16720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:00.801590 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:42823
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:42823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:00.882154 16733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:00.882418 16733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:00.882457 16733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:00.883654 16733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:00.883738 16733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:00.883762 16733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:00.883795 16733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:00.883842 16733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:00.885439 16733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:42823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:42823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:00.885732 16733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:00.885968 16733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:00.890961 16741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:00.890962 16739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:00.891197 16733 server_base.cc:1034] running on GCE node
W20260619 08:51:00.891005 16738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:00.891551 16733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:00.891853 16733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:00.893046 16733 hybrid_clock.cc:648] HybridClock initialized: now 1781859060893030 us; error 31 us; skew 500 ppm
Jun 19 08:51:00 dist-test-slave-p0h9 krb5kdc[16720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859060, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:00.895854 16733 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:00.896335 16733 webserver.cc:460] Webserver started at http://127.15.208.62:34997/ using document root <none> and password file <none>
I20260619 08:51:00.896577 16733 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:00.896649 16733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:00.896819 16733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:00.897918 16733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "ea08867fb29c4a7f8dabfbf281454c4e"
format_stamp: "Formatted at 2026-06-19 08:51:00 on dist-test-slave-p0h9"
server_key: "e193d1a91961d07d2ed1cc1df306e0bf"
server_key_iv: "fd65dcce1a810cd5701c8057d5f63830"
server_key_version: "encryptionkey@0"
I20260619 08:51:00.898288 16733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "ea08867fb29c4a7f8dabfbf281454c4e"
format_stamp: "Formatted at 2026-06-19 08:51:00 on dist-test-slave-p0h9"
server_key: "e193d1a91961d07d2ed1cc1df306e0bf"
server_key_iv: "fd65dcce1a810cd5701c8057d5f63830"
server_key_version: "encryptionkey@0"
I20260619 08:51:00.899633 16733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:00.900349 16748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:00.900555 16733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:00.900668 16733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "ea08867fb29c4a7f8dabfbf281454c4e"
format_stamp: "Formatted at 2026-06-19 08:51:00 on dist-test-slave-p0h9"
server_key: "e193d1a91961d07d2ed1cc1df306e0bf"
server_key_iv: "fd65dcce1a810cd5701c8057d5f63830"
server_key_version: "encryptionkey@0"
I20260619 08:51:00.900771 16733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:00.925734 16733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:00.928334 16733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:00.928521 16733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:00.932261 16733 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:42823
I20260619 08:51:00.932302 16800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:42823 every 8 connection(s)
I20260619 08:51:00.932689 16733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:00.933537 16801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:00.936065 16801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e: Bootstrap starting.
I20260619 08:51:00.936292 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 16733
I20260619 08:51:00.936388 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:00.936595 16192 external_mini_cluster.cc:1428] Setting key cbb9fb83334bfa5704fbe637d92cca95
I20260619 08:51:00.936780 16801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:00.937121 16801 log.cc:824] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:00.937855 16801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e: No bootstrap required, opened a new log
I20260619 08:51:00.940088 16801 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea08867fb29c4a7f8dabfbf281454c4e" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42823 } }
I20260619 08:51:00.940267 16801 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:00.940305 16801 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea08867fb29c4a7f8dabfbf281454c4e, State: Initialized, Role: FOLLOWER
I20260619 08:51:00.940425 16801 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea08867fb29c4a7f8dabfbf281454c4e" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42823 } }
I20260619 08:51:00.940501 16801 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:00.940531 16801 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:00.940574 16801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:00.941270 16801 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea08867fb29c4a7f8dabfbf281454c4e" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42823 } }
I20260619 08:51:00.941357 16801 leader_election.cc:302] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea08867fb29c4a7f8dabfbf281454c4e; no voters:
I20260619 08:51:00.941552 16801 leader_election.cc:288] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:00.941601 16806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:00.941757 16806 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [term 1 LEADER]: Becoming Leader. State: Replica: ea08867fb29c4a7f8dabfbf281454c4e, State: Running, Role: LEADER
I20260619 08:51:00.941821 16801 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:00.941881 16806 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea08867fb29c4a7f8dabfbf281454c4e" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42823 } }
I20260619 08:51:00.942209 16808 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea08867fb29c4a7f8dabfbf281454c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea08867fb29c4a7f8dabfbf281454c4e" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42823 } } }
I20260619 08:51:00.942301 16808 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:00.942530 16809 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea08867fb29c4a7f8dabfbf281454c4e. Latest consensus state: current_term: 1 leader_uuid: "ea08867fb29c4a7f8dabfbf281454c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea08867fb29c4a7f8dabfbf281454c4e" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42823 } } }
I20260619 08:51:00.942615 16809 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e [sys.catalog]: This master's current role is: LEADER
W20260619 08:51:00.943702 16822 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:51:00.943804 16822 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:51:00.943909 16815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:00.944150 16815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:51:00 dist-test-slave-p0h9 krb5kdc[16720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:00.946007 16815 catalog_manager.cc:1347] Generated new cluster ID: d95d10778ee54ff399dc2ce384ad9b3a
I20260619 08:51:00.946060 16815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:00.949024 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:00.937175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43184 (local address 127.15.208.62:42823)
0619 08:51:00.937344 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:00.937355 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:00.937387 (+ 32us) server_negotiation.cc:407] Connection header received
0619 08:51:00.937915 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:00.937923 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:00.938018 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:00.938155 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:00.939015 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:00.939985 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:00.941082 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:00.941595 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:00.941714 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:00.944934 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:00.944938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:00.944939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:00.944951 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:00.947116 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:00.947492 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:00.947495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:00.947495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:00.947593 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:00.947766 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:00.947766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:00.947767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:00.948058 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:00.948202 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:00.948664 (+ 462us) server_negotiation.cc:299] Negotiation successful
0619 08:51:00.948822 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260619 08:51:00.967693 16815 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:00.968462 16815 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:00.973300 16815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea08867fb29c4a7f8dabfbf281454c4e: Generated new TSK 0
I20260619 08:51:00.973498 16815 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:51:01.007364 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42823
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:01.092526 16828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:01.092752 16828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:01.092778 16828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:01.093869 16828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:01.093905 16828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:01.093931 16828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:01.095317 16828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:01.095500 16828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:01.095674 16828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:01.100965 16833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:01.101032 16836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:01.100972 16834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:01.101223 16828 server_base.cc:1034] running on GCE node
I20260619 08:51:01.101455 16828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:01.101653 16828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:01.102821 16828 hybrid_clock.cc:648] HybridClock initialized: now 1781859061102803 us; error 35 us; skew 500 ppm
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:01.105656 16828 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:51:01.106069 16828 webserver.cc:460] Webserver started at http://127.15.208.1:37483/ using document root <none> and password file <none>
I20260619 08:51:01.106282 16828 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:01.106348 16828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:01.106499 16828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:01.107537 16828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "00cb7e41577748678d3e2927d778724a"
format_stamp: "Formatted at 2026-06-19 08:51:01 on dist-test-slave-p0h9"
server_key: "4d27c15d949742a637585d5b9da0e070"
server_key_iv: "3ac75d44c91bbf936186342e3860b122"
server_key_version: "encryptionkey@0"
I20260619 08:51:01.107946 16828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "00cb7e41577748678d3e2927d778724a"
format_stamp: "Formatted at 2026-06-19 08:51:01 on dist-test-slave-p0h9"
server_key: "4d27c15d949742a637585d5b9da0e070"
server_key_iv: "3ac75d44c91bbf936186342e3860b122"
server_key_version: "encryptionkey@0"
I20260619 08:51:01.109337 16828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:01.110050 16843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:01.110215 16828 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:01.110316 16828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "00cb7e41577748678d3e2927d778724a"
format_stamp: "Formatted at 2026-06-19 08:51:01 on dist-test-slave-p0h9"
server_key: "4d27c15d949742a637585d5b9da0e070"
server_key_iv: "3ac75d44c91bbf936186342e3860b122"
server_key_version: "encryptionkey@0"
I20260619 08:51:01.110386 16828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:01.164628 16828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:01.167112 16828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:01.167284 16828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:01.167531 16828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:01.167871 16828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:01.167924 16828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:01.167960 16828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:01.167989 16828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:01.174055 16828 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:44513
I20260619 08:51:01.174130 16957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:44513 every 8 connection(s)
I20260619 08:51:01.174474 16828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:01.182969 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 16828
I20260619 08:51:01.183069 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:01.183264 16192 external_mini_cluster.cc:1428] Setting key 670deb77bebd688c1d727771b78aca5a
I20260619 08:51:01.183463 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.175216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:43943 (local address 127.15.208.62:42823)
0619 08:51:01.175283 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.175285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.175465 (+ 180us) server_negotiation.cc:407] Connection header received
0619 08:51:01.176405 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.176406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.176416 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.176488 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:01.177173 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.177664 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.178620 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.178806 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.178854 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.180923 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:01.180925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:01.180925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:01.180932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:01.182142 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.182501 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.182502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.182502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.182558 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.182760 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.182761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.182761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.182859 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:01.182908 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.183254 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.183361 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:51:01.183652 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.175183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:42823 (local address 127.15.208.1:43943)
0619 08:51:01.175334 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:01.175376 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:01.176294 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:01.176546 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:01.176550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:01.176883 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:01.177086 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.177091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.177745 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.177746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.178541 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.178543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.178875 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.178876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.178982 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.179443 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:01.179452 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:01.180843 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:01.182211 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:01.182216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:01.182218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:01.182426 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:01.182616 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:01.182616 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:01.182616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:01.182708 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:01.182926 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:01.182932 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:01.183101 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 08:51:01.183239 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 08:51:01.183405 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260619 08:51:01.184157 16958 heartbeater.cc:347] Connected to a master server at 127.15.208.62:42823
I20260619 08:51:01.184268 16958 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:01.184440 16958 heartbeater.cc:511] Master 127.15.208.62:42823 requested a full tablet report, sending...
I20260619 08:51:01.184983 16765 ts_manager.cc:194] Registered new tserver with Master: 00cb7e41577748678d3e2927d778724a (127.15.208.1:44513)
I20260619 08:51:01.185879 16765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:43943
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:51:01.208316 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42823
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:01.289124 16963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:01.289309 16963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:01.289331 16963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:01.290496 16963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:01.290540 16963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:01.290565 16963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:01.292896 16963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:01.293171 16963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:01.293403 16963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:01.298372 16968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:01.298349 16969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:01.298349 16971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:01.298591 16963 server_base.cc:1034] running on GCE node
I20260619 08:51:01.298820 16963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:01.299062 16963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:01.300210 16963 hybrid_clock.cc:648] HybridClock initialized: now 1781859061300192 us; error 32 us; skew 500 ppm
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:01.302618 16963 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:51:01.303148 16963 webserver.cc:460] Webserver started at http://127.15.208.2:38913/ using document root <none> and password file <none>
I20260619 08:51:01.303400 16963 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:01.303474 16963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:01.303624 16963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:01.304705 16963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9"
format_stamp: "Formatted at 2026-06-19 08:51:01 on dist-test-slave-p0h9"
server_key: "fad1a37e363e7d514f364d231ab249a1"
server_key_iv: "38949d888be3c2216d2a68474c3c6307"
server_key_version: "encryptionkey@0"
I20260619 08:51:01.305068 16963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9"
format_stamp: "Formatted at 2026-06-19 08:51:01 on dist-test-slave-p0h9"
server_key: "fad1a37e363e7d514f364d231ab249a1"
server_key_iv: "38949d888be3c2216d2a68474c3c6307"
server_key_version: "encryptionkey@0"
I20260619 08:51:01.306272 16963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:01.306947 16978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:01.307126 16963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:01.307235 16963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9"
format_stamp: "Formatted at 2026-06-19 08:51:01 on dist-test-slave-p0h9"
server_key: "fad1a37e363e7d514f364d231ab249a1"
server_key_iv: "38949d888be3c2216d2a68474c3c6307"
server_key_version: "encryptionkey@0"
I20260619 08:51:01.307319 16963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:01.327291 16963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:01.329210 16963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:01.329380 16963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:01.329623 16963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:01.329941 16963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:01.329994 16963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:01.330029 16963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:01.330055 16963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:01.336066 16963 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:37033
I20260619 08:51:01.336117 17091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:37033 every 8 connection(s)
I20260619 08:51:01.336441 16963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:01.343551 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 16963
I20260619 08:51:01.343628 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:01.343802 16192 external_mini_cluster.cc:1428] Setting key d0fb89541c14577b651c67093098638b
I20260619 08:51:01.346189 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.337230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:52483 (local address 127.15.208.62:42823)
0619 08:51:01.337323 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.337344 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.337489 (+ 145us) server_negotiation.cc:407] Connection header received
0619 08:51:01.338355 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.338357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.338366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.338418 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:01.339155 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.339668 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.340656 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.340849 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.340895 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.343040 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:01.343041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:01.343042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:01.343049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:01.344796 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.345234 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.345236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.345236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.345297 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.345548 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.345549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.345550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.345668 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:01.345741 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.346006 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.346097 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 08:51:01.346378 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.337166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:42823 (local address 127.15.208.2:52483)
0619 08:51:01.337351 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:01.337386 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:01.338264 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:01.338515 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:01.338518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:01.338855 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:01.339078 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.339082 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.339754 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.339755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.340580 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.340582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.340910 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.340910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.341009 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.341535 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:01.341543 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:01.342959 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:01.344888 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:01.344895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:01.344898 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:01.345134 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:01.345365 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:01.345366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:01.345366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:01.345480 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:01.345732 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:01.345738 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:01.345876 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 08:51:01.345985 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 08:51:01.346142 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":108,"threads_started":1}
I20260619 08:51:01.346952 17093 heartbeater.cc:347] Connected to a master server at 127.15.208.62:42823
I20260619 08:51:01.347057 17093 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:01.347245 17093 heartbeater.cc:511] Master 127.15.208.62:42823 requested a full tablet report, sending...
I20260619 08:51:01.347640 16765 ts_manager.cc:194] Registered new tserver with Master: cfdc5dec22d245bbbf4c3608e64b2ce9 (127.15.208.2:37033)
I20260619 08:51:01.348008 16765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:52483
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:51:01.369205 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42823
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:01.450487 17098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:01.450703 17098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:01.450727 17098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:01.451798 17098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:01.451833 17098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:01.451858 17098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:01.453291 17098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:01.453466 17098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:01.453644 17098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:01.458453 17106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:01.458480 17103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:01.458475 17104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:01.458765 17098 server_base.cc:1034] running on GCE node
I20260619 08:51:01.458988 17098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:01.459182 17098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:01.460327 17098 hybrid_clock.cc:648] HybridClock initialized: now 1781859061460315 us; error 29 us; skew 500 ppm
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:01.462972 17098 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:51:01.463428 17098 webserver.cc:460] Webserver started at http://127.15.208.3:42281/ using document root <none> and password file <none>
I20260619 08:51:01.463698 17098 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:01.463771 17098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:01.463917 17098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:01.465044 17098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b"
format_stamp: "Formatted at 2026-06-19 08:51:01 on dist-test-slave-p0h9"
server_key: "69d9ab2aa656c4eb76b781ee08c95655"
server_key_iv: "9adab06eaa822c16de676e762b8d8f83"
server_key_version: "encryptionkey@0"
I20260619 08:51:01.465412 17098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b"
format_stamp: "Formatted at 2026-06-19 08:51:01 on dist-test-slave-p0h9"
server_key: "69d9ab2aa656c4eb76b781ee08c95655"
server_key_iv: "9adab06eaa822c16de676e762b8d8f83"
server_key_version: "encryptionkey@0"
I20260619 08:51:01.466709 17098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:01.467417 17113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:01.467633 17098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:01.467731 17098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b"
format_stamp: "Formatted at 2026-06-19 08:51:01 on dist-test-slave-p0h9"
server_key: "69d9ab2aa656c4eb76b781ee08c95655"
server_key_iv: "9adab06eaa822c16de676e762b8d8f83"
server_key_version: "encryptionkey@0"
I20260619 08:51:01.467813 17098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:01.478622 17098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:01.480985 17098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:01.481145 17098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:01.481382 17098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:01.481695 17098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:01.481750 17098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:01.481791 17098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:01.481818 17098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:01.487610 17098 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:35849
I20260619 08:51:01.487655 17226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:35849 every 8 connection(s)
I20260619 08:51:01.488020 17098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:01.493686 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 17098
I20260619 08:51:01.493785 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:01.493952 16192 external_mini_cluster.cc:1428] Setting key 43f381008c7ceec15c9dabc422e37c7f
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:01.497264 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.488678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:39689 (local address 127.15.208.62:42823)
0619 08:51:01.488767 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.488769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.489011 (+ 242us) server_negotiation.cc:407] Connection header received
0619 08:51:01.489977 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.489978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.490002 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.490080 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:01.490725 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.491227 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.492210 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.492402 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.492472 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.494800 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:01.494802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:01.494802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:01.494809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:01.496083 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.496431 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.496431 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.496432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.496478 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.496680 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.496680 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.496680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.496838 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:01.496892 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.497116 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.497195 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:51:01.497507 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.488664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:42823 (local address 127.15.208.3:39689)
0619 08:51:01.488895 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:01.488928 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:01.489861 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:01.490156 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:01.490160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:01.490422 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:01.490622 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.490626 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.491314 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.491315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.492106 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.492107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.492510 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.492511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.492624 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.493137 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:01.493145 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:01.494707 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:01.496156 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:01.496161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:01.496163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:01.496369 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:01.496536 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:01.496537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:01.496537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:01.496628 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:01.496895 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:01.496899 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:01.497016 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 08:51:01.497104 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:51:01.497237 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":147,"threads_started":1}
I20260619 08:51:01.497956 17227 heartbeater.cc:347] Connected to a master server at 127.15.208.62:42823
I20260619 08:51:01.498050 17227 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:01.498235 17227 heartbeater.cc:511] Master 127.15.208.62:42823 requested a full tablet report, sending...
I20260619 08:51:01.498618 16765 ts_manager.cc:194] Registered new tserver with Master: 524b438b8e5c4fd2b8fe59e52a60cb5b (127.15.208.3:35849)
I20260619 08:51:01.498981 16765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:39689
I20260619 08:51:01.505501 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:51:01.513511 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.507051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.15.208.62:42823)
0619 08:51:01.507129 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.507132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.507231 (+ 99us) server_negotiation.cc:407] Connection header received
0619 08:51:01.507308 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.507309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.507321 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.507388 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:01.507950 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.508435 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.509223 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.509407 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.509498 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.510744 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:01.510745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:01.510746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:01.510753 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:01.512246 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.512624 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.512625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.512626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.512681 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.512935 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.512936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.512936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.513097 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:01.513202 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.513308 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.513390 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:51:01.515607 16765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:01.517246 16765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:01.524361 17242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.520101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57186 (local address 127.15.208.3:35849)
0619 08:51:01.520459 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.520462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.520471 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:01.520506 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.520508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.520522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.520594 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:01.520896 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.522155 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.523402 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.523896 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.523994 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.524055 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.524226 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.524272 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":102,"threads_started":1}
I20260619 08:51:01.524348 17239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.519875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:35849 (local address 127.0.0.1:57186)
0619 08:51:01.520028 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:01.520074 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:01.520152 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:01.520711 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:01.520712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:01.520715 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:01.520785 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.520788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.522282 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.522283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.523316 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.523318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.523997 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.523998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.524095 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.524107 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 08:51:01.524207 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 08:51:01.524278 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260619 08:51:01.524798 17241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.519894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:37033 (local address 127.0.0.1:47932)
0619 08:51:01.520692 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:01.520703 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:01.520770 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:01.521016 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:01.521017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:01.521020 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:01.521065 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.521066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.521901 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.521902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.523107 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.523109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.524203 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.524205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.524276 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.524282 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:51:01.524349 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 08:51:01.524671 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260619 08:51:01.525018 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.520169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48706 (local address 127.15.208.1:44513)
0619 08:51:01.520347 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.520351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.520360 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:01.520451 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.520453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.520472 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.520577 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:01.520897 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.522159 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.523556 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.524356 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.524435 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.524489 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.524846 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.524898 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260619 08:51:01.525213 17161 tablet_service.cc:1467] Processing CreateTablet for tablet fbb0f99d164646319c5f1d627216e81f (DEFAULT_TABLE table=test-table [id=36ce99c4a15240559cd54a58ed4072d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:01.524912 17240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.519894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:44513 (local address 127.0.0.1:48706)
0619 08:51:01.520294 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:01.520305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:01.520356 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:01.520666 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:01.520671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:01.520675 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:01.520785 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.520790 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.522304 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.522305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.523442 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.523445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.524533 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.524534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.524653 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.524658 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:51:01.524722 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 08:51:01.524778 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":81,"threads_started":1}
I20260619 08:51:01.525601 17161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbb0f99d164646319c5f1d627216e81f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:01.525830 17244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.520169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.15.208.2:37033)
0619 08:51:01.520883 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.520887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.520896 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:01.520931 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.520933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.520948 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.521027 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:01.521125 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.521797 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.523223 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.524017 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.524102 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.524159 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.525702 (+ 1543us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.525762 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":83,"threads_started":1}
I20260619 08:51:01.525995 16891 tablet_service.cc:1467] Processing CreateTablet for tablet fbb0f99d164646319c5f1d627216e81f (DEFAULT_TABLE table=test-table [id=36ce99c4a15240559cd54a58ed4072d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:01.526321 16891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbb0f99d164646319c5f1d627216e81f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:01.526515 17026 tablet_service.cc:1467] Processing CreateTablet for tablet fbb0f99d164646319c5f1d627216e81f (DEFAULT_TABLE table=test-table [id=36ce99c4a15240559cd54a58ed4072d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:01.526850 17026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbb0f99d164646319c5f1d627216e81f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:01.529067 17245 tablet_bootstrap.cc:492] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b: Bootstrap starting.
I20260619 08:51:01.529805 17246 tablet_bootstrap.cc:492] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a: Bootstrap starting.
I20260619 08:51:01.530000 17245 tablet_bootstrap.cc:654] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:01.530079 17247 tablet_bootstrap.cc:492] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9: Bootstrap starting.
I20260619 08:51:01.530457 17246 tablet_bootstrap.cc:654] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:01.530697 17247 tablet_bootstrap.cc:654] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:01.530825 17245 log.cc:824] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:01.531075 17246 log.cc:824] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:01.531278 17247 log.cc:824] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:01.531797 17245 tablet_bootstrap.cc:492] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b: No bootstrap required, opened a new log
I20260619 08:51:01.531828 17246 tablet_bootstrap.cc:492] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a: No bootstrap required, opened a new log
I20260619 08:51:01.531872 17246 ts_tablet_manager.cc:1397] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:51:01.531872 17245 ts_tablet_manager.cc:1397] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:51:01.532078 17247 tablet_bootstrap.cc:492] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9: No bootstrap required, opened a new log
I20260619 08:51:01.532169 17247 ts_tablet_manager.cc:1397] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:01.533972 17246 raft_consensus.cc:348] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35849 } } peers { permanent_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37033 } } peers { permanent_uuid: "00cb7e41577748678d3e2927d778724a" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44513 } }
I20260619 08:51:01.533949 17245 raft_consensus.cc:348] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35849 } } peers { permanent_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37033 } } peers { permanent_uuid: "00cb7e41577748678d3e2927d778724a" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44513 } }
I20260619 08:51:01.534138 17246 raft_consensus.cc:374] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:01.534138 17245 raft_consensus.cc:374] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:01.534176 17245 raft_consensus.cc:729] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524b438b8e5c4fd2b8fe59e52a60cb5b, State: Initialized, Role: FOLLOWER
I20260619 08:51:01.534176 17246 raft_consensus.cc:729] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cb7e41577748678d3e2927d778724a, State: Initialized, Role: FOLLOWER
I20260619 08:51:01.534032 17247 raft_consensus.cc:348] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35849 } } peers { permanent_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37033 } } peers { permanent_uuid: "00cb7e41577748678d3e2927d778724a" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44513 } }
I20260619 08:51:01.534227 17247 raft_consensus.cc:374] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:01.534267 17247 raft_consensus.cc:729] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfdc5dec22d245bbbf4c3608e64b2ce9, State: Initialized, Role: FOLLOWER
I20260619 08:51:01.534298 17245 consensus_queue.cc:260] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35849 } } peers { permanent_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37033 } } peers { permanent_uuid: "00cb7e41577748678d3e2927d778724a" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44513 } }
I20260619 08:51:01.534312 17246 consensus_queue.cc:260] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35849 } } peers { permanent_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37033 } } peers { permanent_uuid: "00cb7e41577748678d3e2927d778724a" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44513 } }
I20260619 08:51:01.534405 17247 consensus_queue.cc:260] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35849 } } peers { permanent_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37033 } } peers { permanent_uuid: "00cb7e41577748678d3e2927d778724a" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44513 } }
I20260619 08:51:01.534578 17246 ts_tablet_manager.cc:1428] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:51:01.534578 17245 ts_tablet_manager.cc:1428] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:51:01.534700 17227 heartbeater.cc:503] Master 127.15.208.62:42823 was elected leader, sending a full tablet report...
I20260619 08:51:01.534751 17247 ts_tablet_manager.cc:1428] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:51:01.534833 17093 heartbeater.cc:503] Master 127.15.208.62:42823 was elected leader, sending a full tablet report...
I20260619 08:51:01.534948 16958 heartbeater.cc:503] Master 127.15.208.62:42823 was elected leader, sending a full tablet report...
I20260619 08:51:01.540380 17252 raft_consensus.cc:482] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:01.540478 17252 raft_consensus.cc:504] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35849 } } peers { permanent_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37033 } } peers { permanent_uuid: "00cb7e41577748678d3e2927d778724a" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44513 } }
I20260619 08:51:01.540953 17252 leader_election.cc:288] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfdc5dec22d245bbbf4c3608e64b2ce9 (127.15.208.2:37033), 00cb7e41577748678d3e2927d778724a (127.15.208.1:44513)
I20260619 08:51:01.544795 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.541079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:37033 (local address 127.15.208.3:50297)
0619 08:51:01.541126 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:01.541144 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:01.541188 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:01.541424 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:01.541425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:01.541427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:01.541490 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.541492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.542289 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.542291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.543603 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.543606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.544445 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.544446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.544536 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.544540 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:01.544601 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 08:51:01.544656 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 08:51:01.544930 17244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.541269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:50297 (local address 127.15.208.2:37033)
0619 08:51:01.541307 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.541309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.541316 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:51:01.541337 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.541338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.541347 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.541420 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:01.541612 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.542194 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.543736 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.544322 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.544441 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.544451 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.544781 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.544827 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:51:01.545125 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.541195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:60251 (local address 127.15.208.1:44513)
0619 08:51:01.541240 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.541242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.541382 (+ 140us) server_negotiation.cc:407] Connection header received
0619 08:51:01.541501 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.541502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.541512 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.541607 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:01.541797 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.542513 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.544020 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.544656 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.544706 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.544712 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.545009 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.545045 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:51:01.545248 17046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb0f99d164646319c5f1d627216e81f" candidate_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" is_pre_election: true
I20260619 08:51:01.545312 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.541144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:44513 (local address 127.15.208.3:60251)
0619 08:51:01.541297 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:01.541308 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:01.541400 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:01.541612 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:01.541613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:01.541615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:01.541718 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.541720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.542604 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.542604 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.543921 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:01.543924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.544821 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.544822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:01.544930 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.544934 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:01.545210 (+ 276us) client_negotiation.cc:240] Negotiation successful
0619 08:51:01.545249 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260619 08:51:01.545389 17046 raft_consensus.cc:2454] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 524b438b8e5c4fd2b8fe59e52a60cb5b in term 0.
I20260619 08:51:01.545648 17115 leader_election.cc:302] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524b438b8e5c4fd2b8fe59e52a60cb5b, cfdc5dec22d245bbbf4c3608e64b2ce9; no voters:
I20260619 08:51:01.545727 16911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb0f99d164646319c5f1d627216e81f" candidate_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00cb7e41577748678d3e2927d778724a" is_pre_election: true
I20260619 08:51:01.545794 17252 raft_consensus.cc:2781] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:01.545861 17252 raft_consensus.cc:482] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:01.545866 16911 raft_consensus.cc:2454] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 524b438b8e5c4fd2b8fe59e52a60cb5b in term 0.
I20260619 08:51:01.545899 17252 raft_consensus.cc:3037] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:01.546770 17252 raft_consensus.cc:504] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35849 } } peers { permanent_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37033 } } peers { permanent_uuid: "00cb7e41577748678d3e2927d778724a" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44513 } }
I20260619 08:51:01.546952 17252 leader_election.cc:288] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [CANDIDATE]: Term 1 election: Requested vote from peers cfdc5dec22d245bbbf4c3608e64b2ce9 (127.15.208.2:37033), 00cb7e41577748678d3e2927d778724a (127.15.208.1:44513)
I20260619 08:51:01.547097 17046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb0f99d164646319c5f1d627216e81f" candidate_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9"
I20260619 08:51:01.547113 16911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb0f99d164646319c5f1d627216e81f" candidate_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00cb7e41577748678d3e2927d778724a"
I20260619 08:51:01.547186 17046 raft_consensus.cc:3037] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:01.547186 16911 raft_consensus.cc:3037] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:01.548163 17046 raft_consensus.cc:2454] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 524b438b8e5c4fd2b8fe59e52a60cb5b in term 1.
I20260619 08:51:01.548163 16911 raft_consensus.cc:2454] T fbb0f99d164646319c5f1d627216e81f P 00cb7e41577748678d3e2927d778724a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 524b438b8e5c4fd2b8fe59e52a60cb5b in term 1.
I20260619 08:51:01.548350 17115 leader_election.cc:302] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524b438b8e5c4fd2b8fe59e52a60cb5b, cfdc5dec22d245bbbf4c3608e64b2ce9; no voters:
I20260619 08:51:01.548466 17252 raft_consensus.cc:2781] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:01.548625 17252 raft_consensus.cc:686] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [term 1 LEADER]: Becoming Leader. State: Replica: 524b438b8e5c4fd2b8fe59e52a60cb5b, State: Running, Role: LEADER
I20260619 08:51:01.548731 17252 consensus_queue.cc:237] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35849 } } peers { permanent_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37033 } } peers { permanent_uuid: "00cb7e41577748678d3e2927d778724a" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44513 } }
I20260619 08:51:01.549716 16764 catalog_manager.cc:5495] T fbb0f99d164646319c5f1d627216e81f P 524b438b8e5c4fd2b8fe59e52a60cb5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 524b438b8e5c4fd2b8fe59e52a60cb5b (127.15.208.3). New cstate: current_term: 1 leader_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524b438b8e5c4fd2b8fe59e52a60cb5b" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfdc5dec22d245bbbf4c3608e64b2ce9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00cb7e41577748678d3e2927d778724a" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44513 } health_report { overall_health: UNKNOWN } } }
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:51:01.566299 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.558451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48708 (local address 127.15.208.1:44513)
0619 08:51:01.558526 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.558528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.558689 (+ 161us) server_negotiation.cc:407] Connection header received
0619 08:51:01.558770 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.558771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.558781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.558831 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:01.559409 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.559876 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.560577 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.560883 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.560947 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.563554 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:01.563563 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:01.563563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:01.563571 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:01.565271 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.565618 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.565621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.565622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.565679 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.565859 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.565859 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.565860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.566006 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:01.566112 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.566168 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.566230 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859061, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859061, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
W20260619 08:51:01.587037 17094 tablet.cc:2369] T fbb0f99d164646319c5f1d627216e81f P cfdc5dec22d245bbbf4c3608e64b2ce9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:51:01.589478 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.582404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48724 (local address 127.15.208.1:44513)
0619 08:51:01.582474 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.582477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.582563 (+ 86us) server_negotiation.cc:407] Connection header received
0619 08:51:01.582615 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.582616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.582626 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.582681 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:01.583237 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.583760 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.584485 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.584685 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.584833 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.587007 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:01.587023 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:01.587024 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:01.587030 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:01.588453 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.588805 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.588806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.588806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.588855 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.589025 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.589025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.589026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.589113 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:01.589185 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.589322 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.589379 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[16720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:51:01.611418 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:01.604581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48728 (local address 127.15.208.1:44513)
0619 08:51:01.604651 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:01.604655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:01.604865 (+ 210us) server_negotiation.cc:407] Connection header received
0619 08:51:01.604913 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:01.604914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:01.604924 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:01.604967 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:01.605471 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.606032 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.606789 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:01.607021 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:01.607075 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:01.609237 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:01.609239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:01.609239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:01.609246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:01.610508 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.610790 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.610791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.610791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.610837 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:01.610974 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:01.610975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:01.610975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:01.611067 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:01.611131 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:01.611248 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 08:51:01.611345 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 08:51:01.612001 16870 tablet_service.cc:2908] Rejecting scan request for tablet fbb0f99d164646319c5f1d627216e81f: Uninitialized: clean time has not yet been initialized
I20260619 08:51:01.612896 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 16828
I20260619 08:51:01.617918 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 16963
I20260619 08:51:01.622594 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 17098
I20260619 08:51:01.626979 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 16733
2026-06-19T08:51:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1984 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[17269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[17269](info): set up 2 sockets
Jun 19 08:51:01 dist-test-slave-p0h9 krb5kdc[17269](info): commencing operation
krb5kdc: starting...
W20260619 08:51:02.668717 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:02 dist-test-slave-p0h9 krb5kdc[17269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:02.787079 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:44779
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:44779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:02.867043 17282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:02.867256 17282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:02.867278 17282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:02.868456 17282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:02.868505 17282 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 08:51:02.868518 17282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:02.868531 17282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:02.868543 17282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:02.868561 17282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:02.870095 17282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:44779
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:44779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:02.870370 17282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:02.870615 17282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:02.875380 17288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:02.875414 17287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:02.875434 17290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:02.875813 17282 server_base.cc:1034] running on GCE node
I20260619 08:51:02.876066 17282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:02.876325 17282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:02.877465 17282 hybrid_clock.cc:648] HybridClock initialized: now 1781859062877452 us; error 29 us; skew 500 ppm
Jun 19 08:51:02 dist-test-slave-p0h9 krb5kdc[17269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859062, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:02.886945 17282 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:02.887504 17282 webserver.cc:460] Webserver started at http://127.15.208.62:41321/ using document root <none> and password file <none>
I20260619 08:51:02.887760 17282 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:02.887838 17282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:02.888011 17282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:02.889257 17282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "3f01ae1e49dd4050b2ad6ae263cef794"
format_stamp: "Formatted at 2026-06-19 08:51:02 on dist-test-slave-p0h9"
server_key: "17f693713548cce2a637cdf7820fcb5e"
server_key_iv: "c58183443bc144b4666414dc31d39d06"
server_key_version: "encryptionkey@0"
I20260619 08:51:02.889658 17282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "3f01ae1e49dd4050b2ad6ae263cef794"
format_stamp: "Formatted at 2026-06-19 08:51:02 on dist-test-slave-p0h9"
server_key: "17f693713548cce2a637cdf7820fcb5e"
server_key_iv: "c58183443bc144b4666414dc31d39d06"
server_key_version: "encryptionkey@0"
I20260619 08:51:02.891081 17282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:02.891973 17297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:02.892165 17282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:02.892280 17282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "3f01ae1e49dd4050b2ad6ae263cef794"
format_stamp: "Formatted at 2026-06-19 08:51:02 on dist-test-slave-p0h9"
server_key: "17f693713548cce2a637cdf7820fcb5e"
server_key_iv: "c58183443bc144b4666414dc31d39d06"
server_key_version: "encryptionkey@0"
I20260619 08:51:02.892356 17282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:02.900856 17282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:02.903154 17282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:02.903327 17282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:02.908057 17282 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:44779
I20260619 08:51:02.908111 17359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:44779 every 8 connection(s)
I20260619 08:51:02.908424 17282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:02.909087 17360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:02.911345 17360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794: Bootstrap starting.
I20260619 08:51:02.911696 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 17282
I20260619 08:51:02.911780 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:02.911978 16192 external_mini_cluster.cc:1428] Setting key 3ddcb95b1f62e6c88c1de7dda825e174
I20260619 08:51:02.912016 17360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:02.912367 17360 log.cc:824] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:02.913130 17360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794: No bootstrap required, opened a new log
I20260619 08:51:02.914892 17360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f01ae1e49dd4050b2ad6ae263cef794" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44779 } }
I20260619 08:51:02.915050 17360 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:02.915099 17360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f01ae1e49dd4050b2ad6ae263cef794, State: Initialized, Role: FOLLOWER
I20260619 08:51:02.915241 17360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f01ae1e49dd4050b2ad6ae263cef794" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44779 } }
I20260619 08:51:02.915337 17360 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:02.915381 17360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:02.915431 17360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:02.916096 17360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f01ae1e49dd4050b2ad6ae263cef794" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44779 } }
I20260619 08:51:02.916203 17360 leader_election.cc:302] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f01ae1e49dd4050b2ad6ae263cef794; no voters:
I20260619 08:51:02.916359 17360 leader_election.cc:288] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:02.916421 17365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:02.916574 17365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [term 1 LEADER]: Becoming Leader. State: Replica: 3f01ae1e49dd4050b2ad6ae263cef794, State: Running, Role: LEADER
I20260619 08:51:02.916604 17360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:02.916661 17365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f01ae1e49dd4050b2ad6ae263cef794" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44779 } }
I20260619 08:51:02.916982 17367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f01ae1e49dd4050b2ad6ae263cef794. Latest consensus state: current_term: 1 leader_uuid: "3f01ae1e49dd4050b2ad6ae263cef794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f01ae1e49dd4050b2ad6ae263cef794" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44779 } } }
I20260619 08:51:02.916981 17366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f01ae1e49dd4050b2ad6ae263cef794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f01ae1e49dd4050b2ad6ae263cef794" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44779 } } }
I20260619 08:51:02.917160 17366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:02.917356 17367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:02.917507 17373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:02.918222 17373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:02.919874 17373 catalog_manager.cc:1347] Generated new cluster ID: e77c009475b84c4b8af7025fdc3400e0
I20260619 08:51:02.919925 17373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:51:02 dist-test-slave-p0h9 krb5kdc[17269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:02.924265 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:02.912552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35182 (local address 127.15.208.62:44779)
0619 08:51:02.912762 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:02.912772 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:02.912807 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:51:02.913288 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:02.913297 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:02.913360 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:02.913475 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:02.914233 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:02.915616 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:02.916681 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:02.917377 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:02.917463 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:02.920536 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:02.920539 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:02.920540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:02.920552 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:02.922482 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:02.922846 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:02.922848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:02.922849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:02.922940 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:02.923104 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:02.923105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:02.923105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:02.923389 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:02.923531 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:02.923934 (+ 403us) server_negotiation.cc:299] Negotiation successful
0619 08:51:02.924061 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260619 08:51:02.936087 17373 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:02.936640 17373 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:02.942116 17373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794: Generated new TSK 0
I20260619 08:51:02.942314 17373 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:51:02.975260 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44779
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:03.069401 17386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:03.069612 17386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:03.069635 17386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:03.070808 17386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:03.070848 17386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:03.070869 17386 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:51:03.070886 17386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:03.072242 17386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.208.62:44779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:03.072424 17386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:03.072594 17386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:03.077487 17391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:03.077498 17392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:03.077481 17394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:03.077813 17386 server_base.cc:1034] running on GCE node
I20260619 08:51:03.078058 17386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:03.078312 17386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:03.079491 17386 hybrid_clock.cc:648] HybridClock initialized: now 1781859063079472 us; error 49 us; skew 500 ppm
Jun 19 08:51:03 dist-test-slave-p0h9 krb5kdc[17269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859063, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:03.082033 17386 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:51:03.082499 17386 webserver.cc:460] Webserver started at http://127.15.208.1:42147/ using document root <none> and password file <none>
I20260619 08:51:03.082732 17386 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:03.082799 17386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:03.082947 17386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:03.083987 17386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "4a302022ae3b444ca68d78bd294cb7b7"
format_stamp: "Formatted at 2026-06-19 08:51:03 on dist-test-slave-p0h9"
server_key: "614f788f10676b19be066cad4405b61b"
server_key_iv: "f91319fdba606c646a2305fd16efed36"
server_key_version: "encryptionkey@0"
I20260619 08:51:03.084352 17386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "4a302022ae3b444ca68d78bd294cb7b7"
format_stamp: "Formatted at 2026-06-19 08:51:03 on dist-test-slave-p0h9"
server_key: "614f788f10676b19be066cad4405b61b"
server_key_iv: "f91319fdba606c646a2305fd16efed36"
server_key_version: "encryptionkey@0"
I20260619 08:51:03.085799 17386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:03.086562 17401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.086746 17386 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:51:03.086846 17386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "4a302022ae3b444ca68d78bd294cb7b7"
format_stamp: "Formatted at 2026-06-19 08:51:03 on dist-test-slave-p0h9"
server_key: "614f788f10676b19be066cad4405b61b"
server_key_iv: "f91319fdba606c646a2305fd16efed36"
server_key_version: "encryptionkey@0"
I20260619 08:51:03.086925 17386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:03.097119 17386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:03.099267 17386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:03.099427 17386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:03.099969 17386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:03.100025 17386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.100064 17386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:03.100091 17386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:51:03 dist-test-slave-p0h9 krb5kdc[17269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859063, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:03.108301 17386 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:44291
I20260619 08:51:03.108363 17516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:44291 every 8 connection(s)
I20260619 08:51:03.108693 17386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:03.109491 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.100497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44779 (local address 127.15.208.1:41115)
0619 08:51:03.100656 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.100694 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.101210 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.101474 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.101478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.101722 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:03.101897 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.101902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.102633 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.102634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.103497 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.103500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.103904 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.103905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.104058 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.104576 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:03.104584 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:03.106061 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:03.107752 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.107757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.107759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.108125 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.108349 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.108350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.108351 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.108537 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.108879 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:03.108885 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:03.109042 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.109149 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.109306 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260619 08:51:03.109824 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.100511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:41115 (local address 127.15.208.62:44779)
0619 08:51:03.100585 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.100587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.100788 (+ 201us) server_negotiation.cc:407] Connection header received
0619 08:51:03.101334 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.101335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.101347 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.101403 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:03.102002 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.102534 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.103603 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.103810 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.103895 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.106167 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:03.106168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:03.106168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:03.106177 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:03.107689 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.108218 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.108219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.108219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.108284 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.108619 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.108620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.108621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.108823 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:03.108888 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.109630 (+ 742us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.109746 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:51:03.110221 17517 heartbeater.cc:347] Connected to a master server at 127.15.208.62:44779
I20260619 08:51:03.110302 17517 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:03.110481 17517 heartbeater.cc:511] Master 127.15.208.62:44779 requested a full tablet report, sending...
I20260619 08:51:03.110901 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 17386
I20260619 08:51:03.111014 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:03.111027 17313 ts_manager.cc:194] Registered new tserver with Master: 4a302022ae3b444ca68d78bd294cb7b7 (127.15.208.1:44291)
I20260619 08:51:03.111207 16192 external_mini_cluster.cc:1428] Setting key 4b6552a53a4d4133942c46876e2f9c31
I20260619 08:51:03.111824 17313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:41115
I20260619 08:51:03.118726 17523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.111357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44779 (local address 127.15.208.1:59993)
0619 08:51:03.112071 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.112083 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.112137 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.112680 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.112681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.112903 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:03.112956 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.112958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.113950 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.113951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.114476 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.114478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.114995 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.114996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.115080 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.115603 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:03.115610 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:03.115911 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:03.117744 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.117745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.117745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.117910 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.118158 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.118159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.118159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.118238 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.118480 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:03.118481 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:03.118519 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.118573 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.118654 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":167,"threads_started":1}
I20260619 08:51:03.118800 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.111446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:59993 (local address 127.15.208.62:44779)
0619 08:51:03.111528 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.111531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.112123 (+ 592us) server_negotiation.cc:407] Connection header received
0619 08:51:03.112192 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.112192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.112202 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.112250 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:03.113039 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.113817 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.114565 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.114894 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.114977 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.116003 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:03.116004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:03.116004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:03.116012 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:03.117662 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.117985 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.117986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.117986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.118051 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.118282 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.118282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.118283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.118417 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:03.118489 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.118601 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.118686 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:51:03.139289 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44779
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:03.219551 17526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:03.219789 17526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:03.219825 17526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:03.221031 17526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:03.221105 17526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:03.221139 17526 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:51:03.221201 17526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:03.222622 17526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.208.62:44779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:03.222892 17526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:03.223122 17526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:03.228085 17534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:03.228122 17532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:03.228068 17531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:03.228325 17526 server_base.cc:1034] running on GCE node
I20260619 08:51:03.228510 17526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:03.228842 17526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:03.230015 17526 hybrid_clock.cc:648] HybridClock initialized: now 1781859063230001 us; error 45 us; skew 500 ppm
Jun 19 08:51:03 dist-test-slave-p0h9 krb5kdc[17269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859063, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:03.232525 17526 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:51:03.233049 17526 webserver.cc:460] Webserver started at http://127.15.208.2:35209/ using document root <none> and password file <none>
I20260619 08:51:03.233278 17526 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:03.233346 17526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:03.233505 17526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:03.234588 17526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "f56e24980b4b42759fda15071c801eca"
format_stamp: "Formatted at 2026-06-19 08:51:03 on dist-test-slave-p0h9"
server_key: "e9a18444746531066b0b30beb923b862"
server_key_iv: "bd17f5ef8363506e9158b2916d9fe191"
server_key_version: "encryptionkey@0"
I20260619 08:51:03.234951 17526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "f56e24980b4b42759fda15071c801eca"
format_stamp: "Formatted at 2026-06-19 08:51:03 on dist-test-slave-p0h9"
server_key: "e9a18444746531066b0b30beb923b862"
server_key_iv: "bd17f5ef8363506e9158b2916d9fe191"
server_key_version: "encryptionkey@0"
I20260619 08:51:03.236378 17526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:03.237249 17541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.237429 17526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.237527 17526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "f56e24980b4b42759fda15071c801eca"
format_stamp: "Formatted at 2026-06-19 08:51:03 on dist-test-slave-p0h9"
server_key: "e9a18444746531066b0b30beb923b862"
server_key_iv: "bd17f5ef8363506e9158b2916d9fe191"
server_key_version: "encryptionkey@0"
I20260619 08:51:03.237613 17526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:03.264431 17526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:03.266438 17526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:03.266611 17526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:03.267244 17526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:03.267293 17526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.267341 17526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:03.267390 17526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:51:03 dist-test-slave-p0h9 krb5kdc[17269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859063, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:03.274780 17526 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:36915
I20260619 08:51:03.274878 17656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:36915 every 8 connection(s)
I20260619 08:51:03.275214 17526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:51:03.277345 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.267670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:39683 (local address 127.15.208.62:44779)
0619 08:51:03.267740 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.267743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.267992 (+ 249us) server_negotiation.cc:407] Connection header received
0619 08:51:03.268464 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.268466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.268480 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.268576 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:03.269293 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.269811 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.270906 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.271160 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.271248 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.274699 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:03.274700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:03.274701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:03.274710 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:03.276180 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.276559 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.276560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.276560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.276611 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.276862 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.276863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.276863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.276963 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:03.277012 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.277224 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.277284 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:51:03.277523 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.267675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44779 (local address 127.15.208.2:39683)
0619 08:51:03.267851 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.267892 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.268372 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.268632 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.268635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.268992 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:03.269192 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.269197 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.269922 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.269922 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.270812 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.270814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.271255 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.271256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.271446 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.271971 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:03.271982 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:03.274608 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:03.276250 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.276253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.276255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.276458 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.276667 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.276668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.276668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.276812 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.277003 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:03.277006 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:03.277115 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.277184 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.277298 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
I20260619 08:51:03.278033 17657 heartbeater.cc:347] Connected to a master server at 127.15.208.62:44779
I20260619 08:51:03.278115 17657 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:03.278292 17657 heartbeater.cc:511] Master 127.15.208.62:44779 requested a full tablet report, sending...
I20260619 08:51:03.278664 17313 ts_manager.cc:194] Registered new tserver with Master: f56e24980b4b42759fda15071c801eca (127.15.208.2:36915)
I20260619 08:51:03.279163 17313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:39683
I20260619 08:51:03.284067 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 17526
I20260619 08:51:03.284158 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:03.284345 16192 external_mini_cluster.cc:1428] Setting key c38bae6e5e4f1b2c41211a9493099248
I20260619 08:51:03.284627 17663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.278997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44779 (local address 127.15.208.2:37365)
0619 08:51:03.279125 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.279136 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.279198 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.279337 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.279337 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.279510 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:03.279562 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.279563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.280196 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.280197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.280713 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.280715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.281201 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.281203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.281305 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.281793 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:03.281798 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:03.282231 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:03.283567 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.283567 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.283568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.283723 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.283903 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.283904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.283904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.283971 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.284297 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:03.284297 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:03.284343 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.284423 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.284528 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260619 08:51:03.284643 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.279086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:37365 (local address 127.15.208.62:44779)
0619 08:51:03.279125 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.279127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.279188 (+ 61us) server_negotiation.cc:407] Connection header received
0619 08:51:03.279267 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.279267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.279276 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.279320 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:03.279641 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.280101 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.280827 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.281047 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.281138 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.282316 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:03.282317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:03.282318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:03.282323 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:03.283467 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.283786 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.283787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.283787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.283834 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.284037 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.284037 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.284037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.284208 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:03.284291 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.284460 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.284562 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:51:03.307215 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44779
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:03.388149 17666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:03.388350 17666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:03.388373 17666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:03.389614 17666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:03.389655 17666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:03.389676 17666 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:51:03.389802 17666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:03.391268 17666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.208.62:44779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:03.391515 17666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:03.391745 17666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:03.396832 17672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:03.396843 17674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:03.396843 17671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:03.397204 17666 server_base.cc:1034] running on GCE node
I20260619 08:51:03.397421 17666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:03.397667 17666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:03.398837 17666 hybrid_clock.cc:648] HybridClock initialized: now 1781859063398834 us; error 36 us; skew 500 ppm
Jun 19 08:51:03 dist-test-slave-p0h9 krb5kdc[17269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859063, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:03.401350 17666 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:51:03.401795 17666 webserver.cc:460] Webserver started at http://127.15.208.3:44233/ using document root <none> and password file <none>
I20260619 08:51:03.401971 17666 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:03.402010 17666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:03.402105 17666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:03.403092 17666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "e0d0e87b38704dac9919996560dc3a81"
format_stamp: "Formatted at 2026-06-19 08:51:03 on dist-test-slave-p0h9"
server_key: "dd7a75f9ff98b583efdde718a936d906"
server_key_iv: "b54765fffa7f853b76e6ceb687ca8e95"
server_key_version: "encryptionkey@0"
I20260619 08:51:03.403410 17666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "e0d0e87b38704dac9919996560dc3a81"
format_stamp: "Formatted at 2026-06-19 08:51:03 on dist-test-slave-p0h9"
server_key: "dd7a75f9ff98b583efdde718a936d906"
server_key_iv: "b54765fffa7f853b76e6ceb687ca8e95"
server_key_version: "encryptionkey@0"
I20260619 08:51:03.404834 17666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:03.405553 17681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.405727 17666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:03.405795 17666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "e0d0e87b38704dac9919996560dc3a81"
format_stamp: "Formatted at 2026-06-19 08:51:03 on dist-test-slave-p0h9"
server_key: "dd7a75f9ff98b583efdde718a936d906"
server_key_iv: "b54765fffa7f853b76e6ceb687ca8e95"
server_key_version: "encryptionkey@0"
I20260619 08:51:03.405845 17666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:03.431259 17666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:03.433714 17666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:03.433831 17666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:03.434407 17666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:03.434473 17666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.434509 17666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:03.434573 17666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:51:03 dist-test-slave-p0h9 krb5kdc[17269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859063, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:03.441681 17666 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:36761
I20260619 08:51:03.441776 17796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:36761 every 8 connection(s)
I20260619 08:51:03.442262 17666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:03.444388 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.434871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:59565 (local address 127.15.208.62:44779)
0619 08:51:03.434944 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.434946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.435178 (+ 232us) server_negotiation.cc:407] Connection header received
0619 08:51:03.435679 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.435681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.435695 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.435781 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:03.436432 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.437307 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.438353 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.438556 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.438649 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.441839 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:03.441841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:03.441842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:03.441851 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:03.443291 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.443649 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.443650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.443650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.443701 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.443901 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.443902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.443902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.443999 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:03.444046 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.444266 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.444324 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:51:03.444545 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.434869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44779 (local address 127.15.208.3:59565)
0619 08:51:03.435042 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.435084 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.435585 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.435839 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.435842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.436129 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:03.436323 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.436328 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.437424 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.437426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.438255 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.438257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.438687 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.438688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.438813 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.439328 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:03.439337 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:03.441724 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:03.443361 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.443365 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.443366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.443583 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.443758 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.443759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.443759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.443848 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.444051 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:03.444055 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:03.444162 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.444241 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.444354 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260619 08:51:03.445087 17797 heartbeater.cc:347] Connected to a master server at 127.15.208.62:44779
I20260619 08:51:03.445182 17797 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:03.445353 17797 heartbeater.cc:511] Master 127.15.208.62:44779 requested a full tablet report, sending...
I20260619 08:51:03.445734 17313 ts_manager.cc:194] Registered new tserver with Master: e0d0e87b38704dac9919996560dc3a81 (127.15.208.3:36761)
I20260619 08:51:03.446216 17313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:59565
I20260619 08:51:03.451232 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.446079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:53589 (local address 127.15.208.62:44779)
0619 08:51:03.446142 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.446144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.446261 (+ 117us) server_negotiation.cc:407] Connection header received
0619 08:51:03.446321 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.446322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.446333 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.446415 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:03.446711 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.447169 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.447861 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.448068 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.448150 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.449030 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:03.449031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:03.449032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:03.449038 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:03.450104 (+ 1066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.450450 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.450452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.450452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.450503 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.450743 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.450744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.450744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.450851 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:03.450926 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.451073 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.451144 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:51:03.451241 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.446015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44779 (local address 127.15.208.3:53589)
0619 08:51:03.446192 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.446202 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.446265 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.446421 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.446422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.446585 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:03.446634 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.446635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.447260 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.447260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.447777 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.447778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.448153 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.448153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.448227 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.448626 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:03.448630 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:03.448947 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:03.450200 (+ 1253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.450200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.450201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.450366 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.450613 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.450614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.450615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.450672 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.450917 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:03.450918 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:03.450961 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.451049 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.451147 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260619 08:51:03.452042 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 17666
I20260619 08:51:03.452134 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:03.452310 16192 external_mini_cluster.cc:1428] Setting key f7505fd3d5b29fa9c5f7cd32831cf32c
I20260619 08:51:03.453205 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 08:51:03 dist-test-slave-p0h9 krb5kdc[17269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:51:03 dist-test-slave-p0h9 krb5kdc[17269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:03.475178 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.468157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35192 (local address 127.15.208.62:44779)
0619 08:51:03.468231 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.468233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.468243 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:03.468275 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.468275 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.468284 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.468348 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:03.468987 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.469485 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.470238 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.470427 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.470501 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.472824 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:03.472825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:03.472826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:03.472833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:03.473981 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.474363 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.474364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.474364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.474422 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.474654 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.474655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.474656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.474768 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:03.474874 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.474980 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.475073 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:51:03.476842 17313 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:03.478494 17313 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:03.485450 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.481067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.15.208.3:36761)
0619 08:51:03.481468 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.481479 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.481488 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:03.481559 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.481561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.481576 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.481666 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:03.481840 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.483004 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.484167 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.484979 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.485062 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.485136 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.485295 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.485344 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":88,"threads_started":1}
I20260619 08:51:03.485487 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.480892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:44291 (local address 127.0.0.1:58538)
0619 08:51:03.481073 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.481082 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.481122 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.481750 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.481751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.481752 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:03.481794 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.481795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.483090 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.483091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.484176 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.484179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.485172 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.485173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.485277 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.485281 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.485334 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.485376 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":20,"thread_start_us":94,"threads_started":1}
I20260619 08:51:03.485395 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.480873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:36761 (local address 127.0.0.1:47200)
0619 08:51:03.480978 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.481018 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.481087 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.481683 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.481686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.481688 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:03.481757 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.481758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.483125 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.483126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.484077 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.484079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.485070 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.485071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.485159 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.485173 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.485269 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.485317 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":62,"threads_started":1}
I20260619 08:51:03.485967 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.481247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58538 (local address 127.15.208.1:44291)
0619 08:51:03.481443 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.481448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.481457 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:03.481528 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.481530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.481543 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.481642 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:03.481875 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.483019 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.484265 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.484986 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.485087 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.485148 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.485436 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.485484 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260619 08:51:03.486333 17731 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5c95a9ed9a4621af30e5b9f7f54a48 (DEFAULT_TABLE table=test-table [id=a2b3e6e8299449bf98b3a37d54ebe826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:03.486423 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.481283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35068 (local address 127.15.208.2:36915)
0619 08:51:03.481817 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.481821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.481831 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:03.481868 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.481870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.481884 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.481954 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:03.482062 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.482743 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.484816 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.485536 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.485607 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.485662 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.486284 (+ 622us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.486323 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":73,"threads_started":1}
I20260619 08:51:03.486624 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.481089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:36915 (local address 127.0.0.1:35068)
0619 08:51:03.481281 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.481290 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.481339 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.481952 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.481953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.481955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:03.482000 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.482002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.482830 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.482831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.484178 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.484179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.485697 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.485698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.485769 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.485773 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.485819 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.485858 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260619 08:51:03.486690 17731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5c95a9ed9a4621af30e5b9f7f54a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:03.486622 17451 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5c95a9ed9a4621af30e5b9f7f54a48 (DEFAULT_TABLE table=test-table [id=a2b3e6e8299449bf98b3a37d54ebe826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:03.486979 17451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5c95a9ed9a4621af30e5b9f7f54a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:03.487268 17591 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5c95a9ed9a4621af30e5b9f7f54a48 (DEFAULT_TABLE table=test-table [id=a2b3e6e8299449bf98b3a37d54ebe826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:03.487591 17591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5c95a9ed9a4621af30e5b9f7f54a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:03.489715 17820 tablet_bootstrap.cc:492] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: Bootstrap starting.
I20260619 08:51:03.490650 17820 tablet_bootstrap.cc:654] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:03.490809 17821 tablet_bootstrap.cc:492] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: Bootstrap starting.
I20260619 08:51:03.491076 17822 tablet_bootstrap.cc:492] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: Bootstrap starting.
I20260619 08:51:03.491117 17820 log.cc:824] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:03.491500 17821 tablet_bootstrap.cc:654] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:03.491768 17822 tablet_bootstrap.cc:654] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:03.491839 17821 log.cc:824] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:03.492069 17820 tablet_bootstrap.cc:492] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: No bootstrap required, opened a new log
I20260619 08:51:03.492115 17822 log.cc:824] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:03.492134 17820 ts_tablet_manager.cc:1397] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:51:03.492583 17821 tablet_bootstrap.cc:492] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: No bootstrap required, opened a new log
I20260619 08:51:03.492651 17821 ts_tablet_manager.cc:1397] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:03.492897 17822 tablet_bootstrap.cc:492] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: No bootstrap required, opened a new log
I20260619 08:51:03.492976 17822 ts_tablet_manager.cc:1397] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:51:03.494174 17820 raft_consensus.cc:348] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } }
I20260619 08:51:03.494340 17820 raft_consensus.cc:374] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:03.494377 17820 raft_consensus.cc:729] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0d0e87b38704dac9919996560dc3a81, State: Initialized, Role: FOLLOWER
I20260619 08:51:03.494563 17820 consensus_queue.cc:260] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } }
I20260619 08:51:03.494622 17822 raft_consensus.cc:348] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } }
I20260619 08:51:03.494680 17821 raft_consensus.cc:348] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } }
I20260619 08:51:03.494801 17822 raft_consensus.cc:374] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:03.494827 17821 raft_consensus.cc:374] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:03.494840 17820 ts_tablet_manager.cc:1428] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:51:03.494838 17822 raft_consensus.cc:729] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f56e24980b4b42759fda15071c801eca, State: Initialized, Role: FOLLOWER
I20260619 08:51:03.494876 17821 raft_consensus.cc:729] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a302022ae3b444ca68d78bd294cb7b7, State: Initialized, Role: FOLLOWER
I20260619 08:51:03.494952 17797 heartbeater.cc:503] Master 127.15.208.62:44779 was elected leader, sending a full tablet report...
I20260619 08:51:03.494982 17822 consensus_queue.cc:260] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } }
I20260619 08:51:03.494988 17821 consensus_queue.cc:260] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } }
I20260619 08:51:03.495249 17821 ts_tablet_manager.cc:1428] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:51:03.495266 17657 heartbeater.cc:503] Master 127.15.208.62:44779 was elected leader, sending a full tablet report...
I20260619 08:51:03.495370 17517 heartbeater.cc:503] Master 127.15.208.62:44779 was elected leader, sending a full tablet report...
I20260619 08:51:03.495467 17822 ts_tablet_manager.cc:1428] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260619 08:51:03.525774 17658 tablet.cc:2369] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:51:03.609515 17518 tablet.cc:2369] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:51:03.692965 17798 tablet.cc:2369] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:51:03.744097 17828 raft_consensus.cc:482] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:03.744220 17828 raft_consensus.cc:504] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } }
I20260619 08:51:03.744649 17828 leader_election.cc:288] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0d0e87b38704dac9919996560dc3a81 (127.15.208.3:36761), f56e24980b4b42759fda15071c801eca (127.15.208.2:36915)
I20260619 08:51:03.747689 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.744961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:55639 (local address 127.15.208.2:36915)
0619 08:51:03.745006 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.745008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.745019 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:51:03.745122 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.745124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.745133 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.745189 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:03.745353 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.745930 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.746978 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.747337 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.747422 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.747430 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.747584 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.747618 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:51:03.747731 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.745034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:59277 (local address 127.15.208.3:36761)
0619 08:51:03.745087 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.745089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.745097 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:03.745146 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.745147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.745156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.745222 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:03.745397 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.745924 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.746996 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.747368 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.747455 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.747463 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.747628 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.747664 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:51:03.747748 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.744840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:36761 (local address 127.15.208.1:59277)
0619 08:51:03.745017 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.745028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.745083 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.745208 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.745209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.745211 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:03.745312 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.745314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.746023 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.746024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.746913 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.746915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.747454 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.747454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.747536 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.747539 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.747620 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.747660 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":31,"thread_start_us":62,"threads_started":1}
I20260619 08:51:03.748098 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.744833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:36915 (local address 127.15.208.1:55639)
0619 08:51:03.744963 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.744975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.745056 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.745198 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.745199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.745201 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:03.745264 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.745266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.746011 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.746012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.746872 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.746874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.747419 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.747420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.747487 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.747491 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.747577 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.747615 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":54,"thread_start_us":64,"threads_started":1}
I20260619 08:51:03.748070 17751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5c95a9ed9a4621af30e5b9f7f54a48" candidate_uuid: "4a302022ae3b444ca68d78bd294cb7b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0d0e87b38704dac9919996560dc3a81" is_pre_election: true
I20260619 08:51:03.748206 17751 raft_consensus.cc:2454] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a302022ae3b444ca68d78bd294cb7b7 in term 0.
I20260619 08:51:03.748438 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5c95a9ed9a4621af30e5b9f7f54a48" candidate_uuid: "4a302022ae3b444ca68d78bd294cb7b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f56e24980b4b42759fda15071c801eca" is_pre_election: true
I20260619 08:51:03.748450 17403 leader_election.cc:302] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a302022ae3b444ca68d78bd294cb7b7, e0d0e87b38704dac9919996560dc3a81; no voters:
I20260619 08:51:03.748560 17611 raft_consensus.cc:2454] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a302022ae3b444ca68d78bd294cb7b7 in term 0.
I20260619 08:51:03.748616 17828 raft_consensus.cc:2781] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:03.748667 17828 raft_consensus.cc:482] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:03.748697 17828 raft_consensus.cc:3037] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:03.749423 17828 raft_consensus.cc:504] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } }
I20260619 08:51:03.749583 17828 leader_election.cc:288] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [CANDIDATE]: Term 1 election: Requested vote from peers e0d0e87b38704dac9919996560dc3a81 (127.15.208.3:36761), f56e24980b4b42759fda15071c801eca (127.15.208.2:36915)
I20260619 08:51:03.749728 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5c95a9ed9a4621af30e5b9f7f54a48" candidate_uuid: "4a302022ae3b444ca68d78bd294cb7b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f56e24980b4b42759fda15071c801eca"
I20260619 08:51:03.749804 17611 raft_consensus.cc:3037] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:03.750483 17611 raft_consensus.cc:2454] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a302022ae3b444ca68d78bd294cb7b7 in term 1.
I20260619 08:51:03.750582 17751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5c95a9ed9a4621af30e5b9f7f54a48" candidate_uuid: "4a302022ae3b444ca68d78bd294cb7b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0d0e87b38704dac9919996560dc3a81"
I20260619 08:51:03.750684 17751 raft_consensus.cc:3037] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:03.750716 17402 leader_election.cc:302] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a302022ae3b444ca68d78bd294cb7b7, f56e24980b4b42759fda15071c801eca; no voters:
I20260619 08:51:03.750797 17828 raft_consensus.cc:2781] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:03.750936 17828 raft_consensus.cc:686] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 LEADER]: Becoming Leader. State: Replica: 4a302022ae3b444ca68d78bd294cb7b7, State: Running, Role: LEADER
I20260619 08:51:03.751036 17828 consensus_queue.cc:237] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } }
I20260619 08:51:03.751531 17751 raft_consensus.cc:2454] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a302022ae3b444ca68d78bd294cb7b7 in term 1.
I20260619 08:51:03.751822 17313 catalog_manager.cc:5495] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a302022ae3b444ca68d78bd294cb7b7 (127.15.208.1). New cstate: current_term: 1 leader_uuid: "4a302022ae3b444ca68d78bd294cb7b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:03.770000 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.767606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58548 (local address 127.15.208.1:44291)
0619 08:51:03.767673 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.767674 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.767683 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:03.767774 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.767774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.767790 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.767847 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:03.767967 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.768422 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.769237 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.769419 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.769476 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.769679 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:03.769766 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:03.769824 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.769925 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.769943 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:51:03.771835 17611 raft_consensus.cc:1261] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca [term 1 FOLLOWER]: Refusing update from remote peer 4a302022ae3b444ca68d78bd294cb7b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:03.771871 17751 raft_consensus.cc:1261] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81 [term 1 FOLLOWER]: Refusing update from remote peer 4a302022ae3b444ca68d78bd294cb7b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:03.772212 17831 consensus_queue.cc:1035] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:03.772360 17828 consensus_queue.cc:1035] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:03.775110 17835 mvcc.cc:205] Tried to move back new op lower bound from 7298494725206581248 to 7298494725124759552. Current Snapshot: MvccSnapshot[applied={T|T < 7298494725206581248}]
I20260619 08:51:03.775482 17834 mvcc.cc:205] Tried to move back new op lower bound from 7298494725206581248 to 7298494725124759552. Current Snapshot: MvccSnapshot[applied={T|T < 7298494725206581248 or (T in {7298494725206581248})}]
I20260619 08:51:03.778728 17313 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:03.778860 17313 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:03.779676 17313 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794: Sending DeleteTablet for 3 replicas of tablet 0a5c95a9ed9a4621af30e5b9f7f54a48
I20260619 08:51:03.780162 17451 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5c95a9ed9a4621af30e5b9f7f54a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:03 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:58538
I20260619 08:51:03.780241 17731 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5c95a9ed9a4621af30e5b9f7f54a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:03 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:47200
I20260619 08:51:03.780444 17844 tablet_replica.cc:330] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: stopping tablet replica
I20260619 08:51:03.780539 17844 raft_consensus.cc:2229] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:03.780646 17844 raft_consensus.cc:2258] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:03.781097 17844 ts_tablet_manager.cc:1905] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:03.782210 17591 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5c95a9ed9a4621af30e5b9f7f54a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:03 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:35068
I20260619 08:51:03.782506 17845 tablet_replica.cc:330] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: stopping tablet replica
I20260619 08:51:03.782601 17845 raft_consensus.cc:2229] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:03.782697 17845 raft_consensus.cc:2258] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:03.782733 17313 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 08:51:03.782851 17843 tablet_replica.cc:330] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: stopping tablet replica
W20260619 08:51:03.782879 17313 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:03.782951 17843 raft_consensus.cc:2229] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:03.783038 17843 raft_consensus.cc:2258] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:03.783102 17844 ts_tablet_manager.cc:1918] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:03.783165 17844 log.cc:1196] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/0a5c95a9ed9a4621af30e5b9f7f54a48
I20260619 08:51:03.783432 17844 ts_tablet_manager.cc:1939] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P e0d0e87b38704dac9919996560dc3a81: Deleting consensus metadata
I20260619 08:51:03.783449 17845 ts_tablet_manager.cc:1905] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:03.784257 17591 tablet_service.cc:1467] Processing CreateTablet for tablet 128e71e946004492b69724305f3b0fc3 (DEFAULT_TABLE table=test-table [id=5816e77bf9bd46fea721c07493246bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:03.784358 17591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 128e71e946004492b69724305f3b0fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:03.785180 17845 ts_tablet_manager.cc:1918] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:03.785235 17845 log.cc:1196] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/0a5c95a9ed9a4621af30e5b9f7f54a48
I20260619 08:51:03.785460 17845 ts_tablet_manager.cc:1939] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P f56e24980b4b42759fda15071c801eca: Deleting consensus metadata
I20260619 08:51:03.785727 17822 tablet_bootstrap.cc:492] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca: Bootstrap starting.
I20260619 08:51:03.785894 17299 catalog_manager.cc:4841] TS e0d0e87b38704dac9919996560dc3a81 (127.15.208.3:36761): tablet 0a5c95a9ed9a4621af30e5b9f7f54a48 (table test-table [id=a2b3e6e8299449bf98b3a37d54ebe826]) successfully deleted
I20260619 08:51:03.786149 17731 tablet_service.cc:1467] Processing CreateTablet for tablet 128e71e946004492b69724305f3b0fc3 (DEFAULT_TABLE table=test-table [id=5816e77bf9bd46fea721c07493246bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:03.786192 17451 tablet_service.cc:1467] Processing CreateTablet for tablet 128e71e946004492b69724305f3b0fc3 (DEFAULT_TABLE table=test-table [id=5816e77bf9bd46fea721c07493246bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:03.786267 17731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 128e71e946004492b69724305f3b0fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:03.786291 17451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 128e71e946004492b69724305f3b0fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:03.786347 17822 tablet_bootstrap.cc:654] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:03.786787 17843 ts_tablet_manager.cc:1905] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:03.787603 17821 tablet_bootstrap.cc:492] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7: Bootstrap starting.
I20260619 08:51:03.787607 17820 tablet_bootstrap.cc:492] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81: Bootstrap starting.
I20260619 08:51:03.787974 17298 catalog_manager.cc:4841] TS f56e24980b4b42759fda15071c801eca (127.15.208.2:36915): tablet 0a5c95a9ed9a4621af30e5b9f7f54a48 (table test-table [id=a2b3e6e8299449bf98b3a37d54ebe826]) successfully deleted
I20260619 08:51:03.788028 17821 tablet_bootstrap.cc:654] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:03.788028 17820 tablet_bootstrap.cc:654] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:03.788686 17820 tablet_bootstrap.cc:492] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81: No bootstrap required, opened a new log
I20260619 08:51:03.788764 17820 ts_tablet_manager.cc:1397] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:03.788826 17843 ts_tablet_manager.cc:1918] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:03.788873 17821 tablet_bootstrap.cc:492] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7: No bootstrap required, opened a new log
I20260619 08:51:03.788887 17843 log.cc:1196] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/0a5c95a9ed9a4621af30e5b9f7f54a48
I20260619 08:51:03.788915 17821 ts_tablet_manager.cc:1397] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:03.788910 17820 raft_consensus.cc:348] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.788973 17820 raft_consensus.cc:374] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:03.788995 17820 raft_consensus.cc:729] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0d0e87b38704dac9919996560dc3a81, State: Initialized, Role: FOLLOWER
I20260619 08:51:03.789036 17820 consensus_queue.cc:260] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.789057 17821 raft_consensus.cc:348] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.789139 17820 ts_tablet_manager.cc:1428] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.789130 17821 raft_consensus.cc:374] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:03.789194 17821 raft_consensus.cc:729] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a302022ae3b444ca68d78bd294cb7b7, State: Initialized, Role: FOLLOWER
I20260619 08:51:03.789160 17843 ts_tablet_manager.cc:1939] T 0a5c95a9ed9a4621af30e5b9f7f54a48 P 4a302022ae3b444ca68d78bd294cb7b7: Deleting consensus metadata
I20260619 08:51:03.789254 17821 consensus_queue.cc:260] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.789371 17821 ts_tablet_manager.cc:1428] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:03.789744 17298 catalog_manager.cc:4841] TS 4a302022ae3b444ca68d78bd294cb7b7 (127.15.208.1:44291): tablet 0a5c95a9ed9a4621af30e5b9f7f54a48 (table test-table [id=a2b3e6e8299449bf98b3a37d54ebe826]) successfully deleted
I20260619 08:51:03.789989 17822 tablet_bootstrap.cc:492] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca: No bootstrap required, opened a new log
I20260619 08:51:03.790063 17822 ts_tablet_manager.cc:1397] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 08:51:03.790220 17822 raft_consensus.cc:348] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.790292 17822 raft_consensus.cc:374] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:03.790328 17822 raft_consensus.cc:729] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f56e24980b4b42759fda15071c801eca, State: Initialized, Role: FOLLOWER
I20260619 08:51:03.790369 17822 consensus_queue.cc:260] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.790475 17822 ts_tablet_manager.cc:1428] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:03.800504 17826 raft_consensus.cc:482] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:03.800596 17826 raft_consensus.cc:504] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.800910 17826 leader_election.cc:288] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a302022ae3b444ca68d78bd294cb7b7 (127.15.208.1:44291), f56e24980b4b42759fda15071c801eca (127.15.208.2:36915)
I20260619 08:51:03.803972 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.801101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:44291 (local address 127.15.208.3:47383)
0619 08:51:03.801162 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.801174 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.801225 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.801418 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.801418 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.801420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:03.801493 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.801495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.802191 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.802192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.802982 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.802984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.803682 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.803683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.803776 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.803779 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.803852 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.803891 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:51:03.803957 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.801144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:47383 (local address 127.15.208.1:44291)
0619 08:51:03.801199 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.801200 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.801210 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:03.801312 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.801312 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.801323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.801369 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:03.801574 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.802094 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.803069 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.803569 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.803675 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.803684 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.803859 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.803891 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:51:03.804319 17471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128e71e946004492b69724305f3b0fc3" candidate_uuid: "e0d0e87b38704dac9919996560dc3a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a302022ae3b444ca68d78bd294cb7b7" is_pre_election: true
I20260619 08:51:03.804450 17471 raft_consensus.cc:2454] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0d0e87b38704dac9919996560dc3a81 in term 0.
I20260619 08:51:03.804618 17682 leader_election.cc:302] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a302022ae3b444ca68d78bd294cb7b7, e0d0e87b38704dac9919996560dc3a81; no voters:
I20260619 08:51:03.804774 17826 raft_consensus.cc:2781] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:03.804812 17826 raft_consensus.cc:482] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:03.804840 17826 raft_consensus.cc:3037] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:03.805048 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.801295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:59271 (local address 127.15.208.2:36915)
0619 08:51:03.801343 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.801345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.801420 (+ 75us) server_negotiation.cc:407] Connection header received
0619 08:51:03.801491 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.801492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.801501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.801549 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:03.801719 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.802463 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.804049 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.804608 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.804654 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.804661 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.804928 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.804980 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:03.805032 17849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.801196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:36915 (local address 127.15.208.3:59271)
0619 08:51:03.801368 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.801378 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.801430 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.801592 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.801592 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.801593 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:03.801651 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.801652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.802559 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.802560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.803946 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.803949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.804720 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.804754 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.804836 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.804838 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.804913 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.804950 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":54,"threads_started":1}
I20260619 08:51:03.805624 17826 raft_consensus.cc:504] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.805756 17826 leader_election.cc:288] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [CANDIDATE]: Term 1 election: Requested vote from peers 4a302022ae3b444ca68d78bd294cb7b7 (127.15.208.1:44291), f56e24980b4b42759fda15071c801eca (127.15.208.2:36915)
I20260619 08:51:03.806092 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128e71e946004492b69724305f3b0fc3" candidate_uuid: "e0d0e87b38704dac9919996560dc3a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f56e24980b4b42759fda15071c801eca" is_pre_election: true
I20260619 08:51:03.806064 17471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128e71e946004492b69724305f3b0fc3" candidate_uuid: "e0d0e87b38704dac9919996560dc3a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a302022ae3b444ca68d78bd294cb7b7"
I20260619 08:51:03.806164 17471 raft_consensus.cc:3037] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:03.806167 17611 raft_consensus.cc:2454] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0d0e87b38704dac9919996560dc3a81 in term 0.
I20260619 08:51:03.806128 17610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128e71e946004492b69724305f3b0fc3" candidate_uuid: "e0d0e87b38704dac9919996560dc3a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f56e24980b4b42759fda15071c801eca"
I20260619 08:51:03.806208 17610 raft_consensus.cc:3037] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:03.806830 17471 raft_consensus.cc:2454] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0d0e87b38704dac9919996560dc3a81 in term 1.
I20260619 08:51:03.806989 17610 raft_consensus.cc:2454] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0d0e87b38704dac9919996560dc3a81 in term 1.
I20260619 08:51:03.807015 17682 leader_election.cc:302] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a302022ae3b444ca68d78bd294cb7b7, e0d0e87b38704dac9919996560dc3a81; no voters:
I20260619 08:51:03.807134 17826 raft_consensus.cc:2781] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:03.807271 17826 raft_consensus.cc:686] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 1 LEADER]: Becoming Leader. State: Replica: e0d0e87b38704dac9919996560dc3a81, State: Running, Role: LEADER
I20260619 08:51:03.807343 17826 consensus_queue.cc:237] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.807971 17313 catalog_manager.cc:5495] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0d0e87b38704dac9919996560dc3a81 (127.15.208.3). New cstate: current_term: 1 leader_uuid: "e0d0e87b38704dac9919996560dc3a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } health_report { overall_health: HEALTHY } } }
Jun 19 08:51:03 dist-test-slave-p0h9 krb5kdc[17269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859062, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:03.836872 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.829534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35208 (local address 127.15.208.62:44779)
0619 08:51:03.829597 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.829600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.829701 (+ 101us) server_negotiation.cc:407] Connection header received
0619 08:51:03.829742 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.829743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.829754 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.829829 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:03.830263 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.831039 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.831726 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.832029 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.832107 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.834397 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:03.834399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:03.834399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:03.834404 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:03.835675 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.836114 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.836115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.836115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.836178 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.836370 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.836371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.836371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.836463 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:03.836511 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.836688 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.836807 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:51:03.837001 17857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.829450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44779 (local address 127.0.0.1:35208)
0619 08:51:03.829627 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.829638 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.829697 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.829839 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.829840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.830129 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:03.830187 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.830189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.831149 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.831150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.831654 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.831655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.832111 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.832112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.832163 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.832826 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:03.832831 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:03.834310 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:03.835756 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.835764 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.835766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.836027 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.836239 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:03.836240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:03.836240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:03.836310 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:03.836525 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:03.836532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:03.836581 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.836666 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.836898 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":120,"thread_start_us":96,"threads_started":1}
I20260619 08:51:03.839056 17313 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:35208:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 08:51:03.839280 17313 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:03.840758 17451 tablet_service.cc:1467] Processing CreateTablet for tablet d2fc59df029a406bbd1cba418da5b6a5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2b25ef9e32144a11997a9b775d1f7bdd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:51:03.840758 17591 tablet_service.cc:1467] Processing CreateTablet for tablet d2fc59df029a406bbd1cba418da5b6a5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2b25ef9e32144a11997a9b775d1f7bdd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:51:03.840811 17731 tablet_service.cc:1467] Processing CreateTablet for tablet d2fc59df029a406bbd1cba418da5b6a5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2b25ef9e32144a11997a9b775d1f7bdd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:51:03.840907 17591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2fc59df029a406bbd1cba418da5b6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:03.840907 17451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2fc59df029a406bbd1cba418da5b6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:03.840947 17731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2fc59df029a406bbd1cba418da5b6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:03.842435 17820 tablet_bootstrap.cc:492] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: Bootstrap starting.
I20260619 08:51:03.842486 17822 tablet_bootstrap.cc:492] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: Bootstrap starting.
I20260619 08:51:03.842873 17822 tablet_bootstrap.cc:654] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:03.842933 17820 tablet_bootstrap.cc:654] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:03.843236 17821 tablet_bootstrap.cc:492] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: Bootstrap starting.
I20260619 08:51:03.843597 17822 tablet_bootstrap.cc:492] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: No bootstrap required, opened a new log
I20260619 08:51:03.843626 17821 tablet_bootstrap.cc:654] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:03.843667 17822 ts_tablet_manager.cc:1397] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:03.843825 17822 raft_consensus.cc:348] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.843909 17822 raft_consensus.cc:374] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:03.843943 17822 raft_consensus.cc:729] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f56e24980b4b42759fda15071c801eca, State: Initialized, Role: FOLLOWER
I20260619 08:51:03.843989 17822 consensus_queue.cc:260] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.844090 17822 ts_tablet_manager.cc:1428] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.844192 17827 tablet_replica.cc:439] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } }
I20260619 08:51:03.844292 17827 tablet_replica.cc:442] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:03.844533 17821 tablet_bootstrap.cc:492] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: No bootstrap required, opened a new log
I20260619 08:51:03.844580 17821 ts_tablet_manager.cc:1397] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:03.844791 17820 tablet_bootstrap.cc:492] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: No bootstrap required, opened a new log
I20260619 08:51:03.844841 17820 ts_tablet_manager.cc:1397] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:03.844810 17821 raft_consensus.cc:348] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.844880 17821 raft_consensus.cc:374] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:03.844902 17821 raft_consensus.cc:729] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a302022ae3b444ca68d78bd294cb7b7, State: Initialized, Role: FOLLOWER
I20260619 08:51:03.844949 17821 consensus_queue.cc:260] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.844993 17820 raft_consensus.cc:348] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.845028 17821 ts_tablet_manager.cc:1428] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.845073 17820 raft_consensus.cc:374] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:03.845094 17820 raft_consensus.cc:729] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0d0e87b38704dac9919996560dc3a81, State: Initialized, Role: FOLLOWER
I20260619 08:51:03.845050 17831 tablet_replica.cc:439] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } }
I20260619 08:51:03.845139 17831 tablet_replica.cc:442] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:03.845144 17820 consensus_queue.cc:260] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.845229 17820 ts_tablet_manager.cc:1428] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:03.845329 17826 tablet_replica.cc:439] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } }
I20260619 08:51:03.845407 17826 tablet_replica.cc:442] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:03.860376 17827 raft_consensus.cc:482] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:03.860539 17827 raft_consensus.cc:504] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.860946 17827 leader_election.cc:288] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a302022ae3b444ca68d78bd294cb7b7 (127.15.208.1:44291), e0d0e87b38704dac9919996560dc3a81 (127.15.208.3:36761)
I20260619 08:51:03.864813 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.861110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:36761 (local address 127.15.208.2:47637)
0619 08:51:03.861378 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.861388 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.861560 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.861710 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.861711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.861721 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:03.861795 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.861797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.862818 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.862819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.863739 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.863741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.864524 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.864525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.864593 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.864597 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.864664 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.864700 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":85,"threads_started":1}
I20260619 08:51:03.864859 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.861341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:47637 (local address 127.15.208.3:36761)
0619 08:51:03.861384 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.861386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.861532 (+ 146us) server_negotiation.cc:407] Connection header received
0619 08:51:03.861627 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.861629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.861640 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.861704 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:03.861878 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.862712 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.863872 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.864397 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.864487 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.864497 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.864684 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.864758 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:03.865198 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.861266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:56825 (local address 127.15.208.1:44291)
0619 08:51:03.861317 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.861320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.861395 (+ 75us) server_negotiation.cc:407] Connection header received
0619 08:51:03.861533 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.861535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.861545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.861619 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:03.861834 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.862692 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.864149 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.864781 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.864859 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.864869 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.865087 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.865130 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:51:03.865211 17861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.861106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:44291 (local address 127.15.208.2:56825)
0619 08:51:03.861308 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.861323 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.861440 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.861660 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.861661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.861663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:03.861756 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.861759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.862802 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.862804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.864030 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.864033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.864909 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.864910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.864996 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.865001 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.865079 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.865121 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260619 08:51:03.865427 17751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2fc59df029a406bbd1cba418da5b6a5" candidate_uuid: "f56e24980b4b42759fda15071c801eca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0d0e87b38704dac9919996560dc3a81" is_pre_election: true
I20260619 08:51:03.865514 17751 raft_consensus.cc:2454] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f56e24980b4b42759fda15071c801eca in term 0.
I20260619 08:51:03.865752 17471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2fc59df029a406bbd1cba418da5b6a5" candidate_uuid: "f56e24980b4b42759fda15071c801eca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a302022ae3b444ca68d78bd294cb7b7" is_pre_election: true
I20260619 08:51:03.865854 17471 raft_consensus.cc:2454] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f56e24980b4b42759fda15071c801eca in term 0.
I20260619 08:51:03.865840 17543 leader_election.cc:302] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0d0e87b38704dac9919996560dc3a81, f56e24980b4b42759fda15071c801eca; no voters:
I20260619 08:51:03.865965 17827 raft_consensus.cc:2781] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:03.866017 17827 raft_consensus.cc:482] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:03.866055 17827 raft_consensus.cc:3037] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:03.867005 17827 raft_consensus.cc:504] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.867182 17827 leader_election.cc:288] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [CANDIDATE]: Term 1 election: Requested vote from peers 4a302022ae3b444ca68d78bd294cb7b7 (127.15.208.1:44291), e0d0e87b38704dac9919996560dc3a81 (127.15.208.3:36761)
I20260619 08:51:03.867426 17471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2fc59df029a406bbd1cba418da5b6a5" candidate_uuid: "f56e24980b4b42759fda15071c801eca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a302022ae3b444ca68d78bd294cb7b7"
I20260619 08:51:03.867508 17471 raft_consensus.cc:3037] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:03.867646 17751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2fc59df029a406bbd1cba418da5b6a5" candidate_uuid: "f56e24980b4b42759fda15071c801eca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0d0e87b38704dac9919996560dc3a81"
I20260619 08:51:03.867728 17751 raft_consensus.cc:3037] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:03.868371 17471 raft_consensus.cc:2454] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f56e24980b4b42759fda15071c801eca in term 1.
I20260619 08:51:03.868476 17751 raft_consensus.cc:2454] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f56e24980b4b42759fda15071c801eca in term 1.
I20260619 08:51:03.868616 17542 leader_election.cc:302] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a302022ae3b444ca68d78bd294cb7b7, f56e24980b4b42759fda15071c801eca; no voters:
I20260619 08:51:03.868706 17827 raft_consensus.cc:2781] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:03.868906 17827 raft_consensus.cc:686] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [term 1 LEADER]: Becoming Leader. State: Replica: f56e24980b4b42759fda15071c801eca, State: Running, Role: LEADER
I20260619 08:51:03.868989 17827 consensus_queue.cc:237] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } }
I20260619 08:51:03.869402 17863 tablet_replica.cc:439] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: TxnStatusTablet state changed. Reason: New leader f56e24980b4b42759fda15071c801eca. Latest consensus state: current_term: 1 leader_uuid: "f56e24980b4b42759fda15071c801eca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } }
I20260619 08:51:03.869616 17863 tablet_replica.cc:442] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:51:03.869781 17312 catalog_manager.cc:5495] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca reported cstate change: term changed from 0 to 1, leader changed from <none> to f56e24980b4b42759fda15071c801eca (127.15.208.2). New cstate: current_term: 1 leader_uuid: "f56e24980b4b42759fda15071c801eca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:03.869928 17865 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 08:51:03.870033 17865 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 08:51:03.877978 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.875040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35082 (local address 127.15.208.2:36915)
0619 08:51:03.875092 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.875094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.875149 (+ 55us) server_negotiation.cc:407] Connection header received
0619 08:51:03.875262 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.875263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.875282 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.875360 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:03.875519 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.876060 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.877046 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.877272 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.877371 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.877596 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:03.877705 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:03.877802 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.877872 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.877893 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:51:03.878048 17857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.875004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:36915 (local address 127.0.0.1:35082)
0619 08:51:03.875099 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.875111 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.875171 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.875364 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.875365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.875366 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:03.875427 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.875430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.876190 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.876191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.876936 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.876939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.877373 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.877374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.877457 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.877479 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:03.877786 (+ 307us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:03.877797 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.877866 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.877938 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 08:51:03.879205 17751 raft_consensus.cc:1261] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81 [term 1 FOLLOWER]: Refusing update from remote peer f56e24980b4b42759fda15071c801eca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:03.879575 17827 consensus_queue.cc:1035] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:03.879637 17471 raft_consensus.cc:1261] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 FOLLOWER]: Refusing update from remote peer f56e24980b4b42759fda15071c801eca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:03.879953 17827 consensus_queue.cc:1035] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:03.880640 17826 tablet_replica.cc:439] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: TxnStatusTablet state changed. Reason: New leader f56e24980b4b42759fda15071c801eca. Latest consensus state: current_term: 1 leader_uuid: "f56e24980b4b42759fda15071c801eca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } }
I20260619 08:51:03.880823 17826 tablet_replica.cc:442] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:03.881029 17831 tablet_replica.cc:439] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: TxnStatusTablet state changed. Reason: New leader f56e24980b4b42759fda15071c801eca. Latest consensus state: current_term: 1 leader_uuid: "f56e24980b4b42759fda15071c801eca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } }
I20260619 08:51:03.881132 17831 tablet_replica.cc:442] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:03.882221 17863 tablet_replica.cc:439] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f56e24980b4b42759fda15071c801eca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } }
I20260619 08:51:03.882319 17863 tablet_replica.cc:442] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:51:03.882391 17863 tablet_replica.cc:439] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f56e24980b4b42759fda15071c801eca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } }
I20260619 08:51:03.882457 17863 tablet_replica.cc:442] T d2fc59df029a406bbd1cba418da5b6a5 P f56e24980b4b42759fda15071c801eca: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:51:03.882975 17831 tablet_replica.cc:439] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f56e24980b4b42759fda15071c801eca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } }
I20260619 08:51:03.883077 17831 tablet_replica.cc:442] T d2fc59df029a406bbd1cba418da5b6a5 P 4a302022ae3b444ca68d78bd294cb7b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:03.884204 17826 tablet_replica.cc:439] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f56e24980b4b42759fda15071c801eca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 } } peers { permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 } } peers { permanent_uuid: "e0d0e87b38704dac9919996560dc3a81" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 36761 } } }
I20260619 08:51:03.884315 17826 tablet_replica.cc:442] T d2fc59df029a406bbd1cba418da5b6a5 P e0d0e87b38704dac9919996560dc3a81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:03.888101 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.885315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.15.208.3:36761)
0619 08:51:03.885363 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.885366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.885405 (+ 39us) server_negotiation.cc:407] Connection header received
0619 08:51:03.885519 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.885520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.885541 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.885656 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:03.885812 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.886343 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.887186 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.887414 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.887516 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.887707 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:03.887829 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:03.887936 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.887983 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.888004 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:51:03.893625 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.890892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:48997 (local address 127.15.208.2:36915)
0619 08:51:03.890946 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.890948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.891002 (+ 54us) server_negotiation.cc:407] Connection header received
0619 08:51:03.891108 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.891109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.891119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.891203 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:03.891367 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.891878 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.892766 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.893012 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.893112 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.893311 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:03.893346 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:03.893445 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.893522 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.893543 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:03.893637 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.890847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:36915 (local address 127.15.208.3:48997)
0619 08:51:03.890920 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.890934 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.891012 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.891205 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.891207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.891208 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:03.891273 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.891275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.891994 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.891995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.892632 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.892634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.893114 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.893115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.893185 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.893208 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:03.893448 (+ 240us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:03.893451 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.893516 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.893563 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 08:51:03.896128 17471 raft_consensus.cc:1261] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 FOLLOWER]: Refusing update from remote peer e0d0e87b38704dac9919996560dc3a81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:03.896149 17610 raft_consensus.cc:1261] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca [term 1 FOLLOWER]: Refusing update from remote peer e0d0e87b38704dac9919996560dc3a81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:03.896512 17850 consensus_queue.cc:1035] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f56e24980b4b42759fda15071c801eca" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:03.896641 17826 consensus_queue.cc:1035] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a302022ae3b444ca68d78bd294cb7b7" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:03.902156 17836 mvcc.cc:205] Tried to move back new op lower bound from 7298494725716680704 to 7298494725355372544. Current Snapshot: MvccSnapshot[applied={T|T < 7298494725716680704}]
I20260619 08:51:03.911401 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.907158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:39305 (local address 127.15.208.3:36761)
0619 08:51:03.907200 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.907202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.907322 (+ 120us) server_negotiation.cc:407] Connection header received
0619 08:51:03.907384 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.907385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.907395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.907450 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:03.907605 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.908243 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.910053 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.910346 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.910385 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.911116 (+ 731us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:03.911152 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:03.911181 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.911290 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.911313 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:51:03.911662 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.906962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:36761 (local address 127.15.208.2:39305)
0619 08:51:03.907251 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:03.907263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:03.907336 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:03.907455 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:03.907455 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:03.907457 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:03.907528 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.907530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.908338 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.908339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.909942 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:03.909945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.910439 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.910440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:03.910500 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.910515 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:03.911223 (+ 708us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:03.911225 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:03.911514 (+ 289us) client_negotiation.cc:240] Negotiation successful
0619 08:51:03.911564 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":94,"threads_started":1}
I20260619 08:51:03.922785 17312 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:03.922919 17312 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:03.923601 17312 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f01ae1e49dd4050b2ad6ae263cef794: Sending DeleteTablet for 3 replicas of tablet 128e71e946004492b69724305f3b0fc3
I20260619 08:51:03.923884 17451 tablet_service.cc:1514] Processing DeleteTablet for tablet 128e71e946004492b69724305f3b0fc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:03 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:58538
I20260619 08:51:03.923992 17843 tablet_replica.cc:330] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7: stopping tablet replica
I20260619 08:51:03.924044 17843 raft_consensus.cc:2229] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:03.924084 17843 raft_consensus.cc:2258] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:03.924366 17731 tablet_service.cc:1514] Processing DeleteTablet for tablet 128e71e946004492b69724305f3b0fc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:03 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:47200
I20260619 08:51:03.924522 17844 tablet_replica.cc:330] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81: stopping tablet replica
I20260619 08:51:03.924603 17844 raft_consensus.cc:2229] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:03.924708 17844 raft_consensus.cc:2258] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:03.924706 17843 ts_tablet_manager.cc:1905] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:03.925038 17844 ts_tablet_manager.cc:1905] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:03.926448 17843 ts_tablet_manager.cc:1918] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:03.926520 17843 log.cc:1196] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/128e71e946004492b69724305f3b0fc3
I20260619 08:51:03.926621 17844 ts_tablet_manager.cc:1918] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:03.926682 17844 log.cc:1196] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/128e71e946004492b69724305f3b0fc3
I20260619 08:51:03.926750 17843 ts_tablet_manager.cc:1939] T 128e71e946004492b69724305f3b0fc3 P 4a302022ae3b444ca68d78bd294cb7b7: Deleting consensus metadata
I20260619 08:51:03.926889 17844 ts_tablet_manager.cc:1939] T 128e71e946004492b69724305f3b0fc3 P e0d0e87b38704dac9919996560dc3a81: Deleting consensus metadata
I20260619 08:51:03.927207 17299 catalog_manager.cc:4841] TS e0d0e87b38704dac9919996560dc3a81 (127.15.208.3:36761): tablet 128e71e946004492b69724305f3b0fc3 (table test-table [id=5816e77bf9bd46fea721c07493246bec]) successfully deleted
I20260619 08:51:03.927589 17591 tablet_service.cc:1514] Processing DeleteTablet for tablet 128e71e946004492b69724305f3b0fc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:03 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:35068
I20260619 08:51:03.927688 17845 tablet_replica.cc:330] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca: stopping tablet replica
I20260619 08:51:03.927739 17845 raft_consensus.cc:2229] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:03.927784 17845 raft_consensus.cc:2258] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:03.928071 17298 catalog_manager.cc:4841] TS 4a302022ae3b444ca68d78bd294cb7b7 (127.15.208.1:44291): tablet 128e71e946004492b69724305f3b0fc3 (table test-table [id=5816e77bf9bd46fea721c07493246bec]) successfully deleted
I20260619 08:51:03.928355 17845 ts_tablet_manager.cc:1905] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:03.929900 17845 ts_tablet_manager.cc:1918] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:03.929973 17845 log.cc:1196] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/128e71e946004492b69724305f3b0fc3
I20260619 08:51:03.930203 17845 ts_tablet_manager.cc:1939] T 128e71e946004492b69724305f3b0fc3 P f56e24980b4b42759fda15071c801eca: Deleting consensus metadata
I20260619 08:51:03.930615 17298 catalog_manager.cc:4841] TS f56e24980b4b42759fda15071c801eca (127.15.208.2:36915): tablet 128e71e946004492b69724305f3b0fc3 (table test-table [id=5816e77bf9bd46fea721c07493246bec]) successfully deleted
Jun 19 08:51:03 dist-test-slave-p0h9 krb5kdc[17269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:51:03.943639 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.933052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58550 (local address 127.15.208.1:44291)
0619 08:51:03.933097 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.933099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.933362 (+ 263us) server_negotiation.cc:407] Connection header received
0619 08:51:03.933466 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.933467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.933478 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.933509 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:03.934305 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.935058 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.936139 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.936444 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.936486 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.940227 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:03.940236 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:03.940237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:03.940244 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:03.942195 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.942665 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.942668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.942669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.942734 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.942971 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.942972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.942972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.943136 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:03.943292 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.943440 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.943536 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 08:51:03.944386 17511 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58550
I20260619 08:51:03.957139 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:03.949067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35212 (local address 127.15.208.62:44779)
0619 08:51:03.949117 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:03.949119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:03.949182 (+ 63us) server_negotiation.cc:407] Connection header received
0619 08:51:03.949248 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:03.949248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:03.949258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:03.949319 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:03.950009 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.950783 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.951916 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:03.952237 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:03.952309 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:03.954069 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:03.954071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:03.954071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:03.954079 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:03.955832 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.956261 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.956262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.956262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.956323 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:03.956547 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:03.956548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:03.956549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:03.956681 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:03.956795 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:03.956944 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 08:51:03.957032 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260619 08:51:03.957340 17312 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35212
I20260619 08:51:03.958181 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 17386
I20260619 08:51:03.967223 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 17526
I20260619 08:51:03.972441 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 17666
I20260619 08:51:03.977007 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 17282
2026-06-19T08:51:03Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2351 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:04 dist-test-slave-p0h9 krb5kdc[17886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:04 dist-test-slave-p0h9 krb5kdc[17886](info): set up 2 sockets
Jun 19 08:51:04 dist-test-slave-p0h9 krb5kdc[17886](info): commencing operation
krb5kdc: starting...
W20260619 08:51:05.028919 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:05.148047 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:35983
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:33731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:35983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:05.228183 17899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:05.228399 17899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:05.228421 17899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:05.229628 17899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:05.229674 17899 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 08:51:05.229688 17899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:05.229701 17899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:05.229714 17899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:05.229772 17899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:05.231230 17899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:33731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:35983
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:35983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:05.231488 17899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:05.231741 17899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:05.239164 17904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:05.239207 17905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:05.239171 17907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:05.239624 17899 server_base.cc:1034] running on GCE node
I20260619 08:51:05.239881 17899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:05.240216 17899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:05.241387 17899 hybrid_clock.cc:648] HybridClock initialized: now 1781859065241373 us; error 31 us; skew 500 ppm
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:05.244073 17899 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:05.244580 17899 webserver.cc:460] Webserver started at http://127.15.208.62:36287/ using document root <none> and password file <none>
I20260619 08:51:05.244869 17899 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:05.244937 17899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:05.245083 17899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:05.246390 17899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "d6e0bade3ff14daeb99bb991e6eeb285"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "3b5d1158ed6da695222d4a831e1e7417"
server_key_iv: "91dfe1b147d3fd3d6b686fb13e7606a7"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.246711 17899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "d6e0bade3ff14daeb99bb991e6eeb285"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "3b5d1158ed6da695222d4a831e1e7417"
server_key_iv: "91dfe1b147d3fd3d6b686fb13e7606a7"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.247990 17899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:05.248766 17914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:05.248948 17899 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:05.249049 17899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "d6e0bade3ff14daeb99bb991e6eeb285"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "3b5d1158ed6da695222d4a831e1e7417"
server_key_iv: "91dfe1b147d3fd3d6b686fb13e7606a7"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.249140 17899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:05.265252 17899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:05.267257 17899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:05.267431 17899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:05.272261 17899 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:35983
I20260619 08:51:05.272315 17976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:35983 every 8 connection(s)
I20260619 08:51:05.272650 17899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:05.272913 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 17899
I20260619 08:51:05.273031 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:05.273218 16192 external_mini_cluster.cc:1428] Setting key 11773b72c7478cbf080760a934345e3d
I20260619 08:51:05.273518 17977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:05.276873 17977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285: Bootstrap starting.
I20260619 08:51:05.277644 17977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:05.277988 17977 log.cc:824] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:05.278764 17977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285: No bootstrap required, opened a new log
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:05.280644 17977 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e0bade3ff14daeb99bb991e6eeb285" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35983 } }
I20260619 08:51:05.280884 17977 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:05.280938 17977 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e0bade3ff14daeb99bb991e6eeb285, State: Initialized, Role: FOLLOWER
I20260619 08:51:05.281091 17977 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e0bade3ff14daeb99bb991e6eeb285" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35983 } }
I20260619 08:51:05.281188 17977 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:05.281227 17977 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:05.281280 17977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:05.282078 17977 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e0bade3ff14daeb99bb991e6eeb285" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35983 } }
I20260619 08:51:05.282195 17977 leader_election.cc:302] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6e0bade3ff14daeb99bb991e6eeb285; no voters:
I20260619 08:51:05.283557 17977 leader_election.cc:288] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:05.283591 17982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:05.283707 17982 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [term 1 LEADER]: Becoming Leader. State: Replica: d6e0bade3ff14daeb99bb991e6eeb285, State: Running, Role: LEADER
I20260619 08:51:05.283792 17982 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e0bade3ff14daeb99bb991e6eeb285" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35983 } }
I20260619 08:51:05.283859 17977 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:05.284107 17982 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6e0bade3ff14daeb99bb991e6eeb285. Latest consensus state: current_term: 1 leader_uuid: "d6e0bade3ff14daeb99bb991e6eeb285" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e0bade3ff14daeb99bb991e6eeb285" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35983 } } }
I20260619 08:51:05.284180 17982 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:05.284118 17983 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6e0bade3ff14daeb99bb991e6eeb285" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e0bade3ff14daeb99bb991e6eeb285" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35983 } } }
I20260619 08:51:05.284232 17983 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:05.285256 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.273838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.15.208.62:35983)
0619 08:51:05.274037 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.274047 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.274081 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:51:05.274709 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.274719 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.274783 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.274940 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:05.275630 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.277043 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.277932 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.278487 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.278596 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.281081 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:05.281085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:05.281085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:05.281097 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:05.283193 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.283658 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.283661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.283662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.283781 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.283981 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.283982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.283982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.284311 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:05.284453 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.284869 (+ 416us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.285036 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
W20260619 08:51:05.285702 17998 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:51:05.285760 17998 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:51:05.285809 17986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:05.285949 17986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:05.287431 17986 catalog_manager.cc:1347] Generated new cluster ID: 6ccb167e201e47b984fd1bac3fb35683
I20260619 08:51:05.287490 17986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:05.305074 17986 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:05.305770 17986 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:05.313071 17986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285: Generated new TSK 0
I20260619 08:51:05.313316 17986 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:51:05.351508 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:35983
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:33731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:05.441879 18003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:05.442095 18003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:05.442123 18003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:05.443243 18003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:05.443281 18003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:05.443302 18003 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 08:51:05.443315 18003 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:51:05.443332 18003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:05.444882 18003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:33731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.208.62:35983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:05.445086 18003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:05.445271 18003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:05.450197 18008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:05.450237 18011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:05.450253 18009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:05.450493 18003 server_base.cc:1034] running on GCE node
I20260619 08:51:05.450686 18003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:05.450961 18003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:05.452113 18003 hybrid_clock.cc:648] HybridClock initialized: now 1781859065452090 us; error 43 us; skew 500 ppm
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:05.454871 18003 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:51:05.455394 18003 webserver.cc:460] Webserver started at http://127.15.208.1:46467/ using document root <none> and password file <none>
I20260619 08:51:05.455647 18003 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:05.455698 18003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:05.455881 18003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:05.457171 18003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "abd93b64f7804f70a89e70fe2c90be39"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "6a8e51d05c0092af9f7547d1fd460484"
server_key_iv: "aa04488953e7cd6544cc841ff77544b5"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.457566 18003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "abd93b64f7804f70a89e70fe2c90be39"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "6a8e51d05c0092af9f7547d1fd460484"
server_key_iv: "aa04488953e7cd6544cc841ff77544b5"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.458998 18003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:05.459792 18018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:05.460000 18003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:05.460127 18003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "abd93b64f7804f70a89e70fe2c90be39"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "6a8e51d05c0092af9f7547d1fd460484"
server_key_iv: "aa04488953e7cd6544cc841ff77544b5"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.460214 18003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:05.483980 18003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:05.486203 18003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:05.486372 18003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:05.486881 18003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:05.486944 18003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:05.487006 18003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:05.487036 18003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:05.494527 18003 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:35061
I20260619 08:51:05.494633 18133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:35061 every 8 connection(s)
I20260619 08:51:05.494954 18003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:05.497174 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.487523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:48645 (local address 127.15.208.62:35983)
0619 08:51:05.487578 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.487582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.487808 (+ 226us) server_negotiation.cc:407] Connection header received
0619 08:51:05.488351 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.488352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.488366 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.488414 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:05.489257 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.489761 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.490836 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.491115 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.491201 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.494399 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:05.494401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:05.494401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:05.494410 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:05.495901 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.496271 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.496272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.496272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.496324 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.496530 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.496531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.496531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.496666 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:05.496719 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.496993 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.497062 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:51:05.497326 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.487509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:35983 (local address 127.15.208.1:48645)
0619 08:51:05.487685 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:05.487721 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:05.488236 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:05.488487 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:05.488492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:05.488930 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:05.489141 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.489147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.489855 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.489856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.490740 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.490743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.491211 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.491212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.491338 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.491941 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:05.491952 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:05.494280 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:05.495972 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.495975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.495977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.496201 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.496382 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.496382 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.496383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.496475 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.496757 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:05.496762 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:05.496894 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 08:51:05.496979 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 08:51:05.497121 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":115,"threads_started":1}
I20260619 08:51:05.497326 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 18003
I20260619 08:51:05.497418 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:05.497608 16192 external_mini_cluster.cc:1428] Setting key 40a47bfa762ab885b55f6dfbd76c2eae
I20260619 08:51:05.497961 18134 heartbeater.cc:347] Connected to a master server at 127.15.208.62:35983
I20260619 08:51:05.498050 18134 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:05.498234 18134 heartbeater.cc:511] Master 127.15.208.62:35983 requested a full tablet report, sending...
I20260619 08:51:05.498781 17930 ts_manager.cc:194] Registered new tserver with Master: abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061)
I20260619 08:51:05.499708 17930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:48645
I20260619 08:51:05.506357 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.500149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:35983 (local address 127.15.208.1:46471)
0619 08:51:05.500281 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:05.500291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:05.500374 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:05.500493 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:05.500494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:05.500686 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:05.500783 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.500785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.501701 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.501702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.502203 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.502205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.502684 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.502684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.502739 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.503198 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:05.503202 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:05.503481 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:05.505436 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.505437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.505437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.505597 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.505800 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.505801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.505801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.505849 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.506128 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:05.506129 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:05.506163 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 08:51:05.506212 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:51:05.506290 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260619 08:51:05.506453 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.500188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:46471 (local address 127.15.208.62:35983)
0619 08:51:05.500228 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.500230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.500361 (+ 131us) server_negotiation.cc:407] Connection header received
0619 08:51:05.500422 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.500423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.500433 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.500488 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:05.500843 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.501608 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.502286 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.502603 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.502683 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.503572 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:05.503573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:05.503574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:05.503582 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:05.505359 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.505675 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.505677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.505677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.505735 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.505910 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.505910 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.505911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.506061 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:05.506134 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.506241 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.506327 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:51:05.523599 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:35983
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:33731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:05.605219 18143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:05.605433 18143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:05.605459 18143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:05.606559 18143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:05.606594 18143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:05.606614 18143 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 08:51:05.606626 18143 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:51:05.606642 18143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:05.608023 18143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:33731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.208.62:35983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:05.608209 18143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:05.608395 18143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:05.613461 18151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:05.613467 18149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:05.613476 18148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:05.613674 18143 server_base.cc:1034] running on GCE node
I20260619 08:51:05.613866 18143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:05.614097 18143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:05.615244 18143 hybrid_clock.cc:648] HybridClock initialized: now 1781859065615232 us; error 28 us; skew 500 ppm
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:05.617756 18143 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:51:05.618234 18143 webserver.cc:460] Webserver started at http://127.15.208.2:37989/ using document root <none> and password file <none>
I20260619 08:51:05.618460 18143 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:05.618505 18143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:05.618655 18143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:05.619700 18143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "e6ad6e382e7e4f9c85e64df427df77f1"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "cf35586eb4e41c27d19b17afc5043c67"
server_key_iv: "b64a8208fe95d7064d7e09df4ef9cc9e"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.620081 18143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "e6ad6e382e7e4f9c85e64df427df77f1"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "cf35586eb4e41c27d19b17afc5043c67"
server_key_iv: "b64a8208fe95d7064d7e09df4ef9cc9e"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.621462 18143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:05.622162 18158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:05.622350 18143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:05.622440 18143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "e6ad6e382e7e4f9c85e64df427df77f1"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "cf35586eb4e41c27d19b17afc5043c67"
server_key_iv: "b64a8208fe95d7064d7e09df4ef9cc9e"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.622493 18143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:05.642531 18143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:05.645008 18143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:05.645135 18143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:05.645610 18143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:05.645649 18143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:05.645704 18143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:05.645731 18143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:05.652588 18143 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:36511
I20260619 08:51:05.652654 18273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:36511 every 8 connection(s)
I20260619 08:51:05.653036 18143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:05.656312 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.646133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:35569 (local address 127.15.208.62:35983)
0619 08:51:05.646204 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.646206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.646400 (+ 194us) server_negotiation.cc:407] Connection header received
0619 08:51:05.647050 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.647052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.647067 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.647157 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:05.648002 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.648893 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.650537 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.650862 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.650948 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.653683 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:05.653685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:05.653685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:05.653692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:05.655143 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.655501 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.655502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.655502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.655576 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.655802 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.655803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.655803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.655909 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:05.655963 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.656188 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.656250 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:51:05.656517 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.646121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:35983 (local address 127.15.208.2:35569)
0619 08:51:05.646270 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:05.646311 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:05.646939 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:05.647217 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:05.647220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:05.647592 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:05.647881 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.647888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.649003 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.649005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.650411 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.650414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.650950 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.650951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.651104 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.651813 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:05.651824 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:05.653591 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:05.655222 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.655228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.655229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.655435 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.655651 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.655651 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.655651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.655745 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.655966 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:05.655970 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:05.656080 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 08:51:05.656160 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:51:05.656277 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260619 08:51:05.657109 18274 heartbeater.cc:347] Connected to a master server at 127.15.208.62:35983
I20260619 08:51:05.657191 18274 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:05.657369 18274 heartbeater.cc:511] Master 127.15.208.62:35983 requested a full tablet report, sending...
I20260619 08:51:05.657799 17931 ts_manager.cc:194] Registered new tserver with Master: e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511)
I20260619 08:51:05.658313 17931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:35569
I20260619 08:51:05.658556 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 18143
I20260619 08:51:05.658669 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:05.658874 16192 external_mini_cluster.cc:1428] Setting key e51f72449ece360dfbb13d85ef2e164d
I20260619 08:51:05.664376 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.658180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:41603 (local address 127.15.208.62:35983)
0619 08:51:05.658214 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.658216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.658375 (+ 159us) server_negotiation.cc:407] Connection header received
0619 08:51:05.658458 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.658460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.658472 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.658506 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:05.659032 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.659607 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.660652 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.660944 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.661017 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.661904 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:05.661905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:05.661905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:05.661910 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:05.663246 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.663669 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.663670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.663670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.663719 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.663907 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.663907 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.663907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.664027 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:05.664082 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.664261 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.664317 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:51:05.664508 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.658115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:35983 (local address 127.15.208.2:41603)
0619 08:51:05.658276 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:05.658293 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:05.658343 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:05.658582 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:05.658583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:05.658854 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:05.658922 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.658924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.659708 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.659709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.660574 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.660577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.661024 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.661025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.661095 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.661541 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:05.661545 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:05.661826 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:05.663342 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.663343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.663344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.663599 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.663779 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.663780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.663780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.663850 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.664104 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:05.664105 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:05.664158 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 08:51:05.664247 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 08:51:05.664360 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:51:05.684376 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:35983
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:33731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:05.766749 18283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:05.766957 18283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:05.766983 18283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:05.768139 18283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:05.768183 18283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:05.768204 18283 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 08:51:05.768216 18283 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:51:05.768280 18283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:05.770217 18283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:33731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.208.62:35983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:05.770478 18283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:05.770735 18283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:05.776535 18288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:05.776794 18291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:05.776906 18289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:05.780776 18283 server_base.cc:1034] running on GCE node
I20260619 08:51:05.781046 18283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:05.781344 18283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:05.782518 18283 hybrid_clock.cc:648] HybridClock initialized: now 1781859065782497 us; error 42 us; skew 500 ppm
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:05.786064 18283 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:51:05.786635 18283 webserver.cc:460] Webserver started at http://127.15.208.3:37225/ using document root <none> and password file <none>
I20260619 08:51:05.786904 18283 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:05.786969 18283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:05.787106 18283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:05.788630 18283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "831b3bcb69b3453eb9d1ed8566176761"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "5aa39b8fefeafd371088cccdb4561c61"
server_key_iv: "420e86277722417eb701ff42ce5e8495"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.789076 18283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "831b3bcb69b3453eb9d1ed8566176761"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "5aa39b8fefeafd371088cccdb4561c61"
server_key_iv: "420e86277722417eb701ff42ce5e8495"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.790977 18283 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 08:51:05.791827 18298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:05.792023 18283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:05.792115 18283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "831b3bcb69b3453eb9d1ed8566176761"
format_stamp: "Formatted at 2026-06-19 08:51:05 on dist-test-slave-p0h9"
server_key: "5aa39b8fefeafd371088cccdb4561c61"
server_key_iv: "420e86277722417eb701ff42ce5e8495"
server_key_version: "encryptionkey@0"
I20260619 08:51:05.792184 18283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:05.816301 18283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:05.818897 18283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:05.819060 18283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:05.819721 18283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:05.819780 18283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:05.819815 18283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:05.819844 18283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:05.827793 18283 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:40081
I20260619 08:51:05.827850 18413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:40081 every 8 connection(s)
I20260619 08:51:05.828224 18283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:05.830973 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.820109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:35267 (local address 127.15.208.62:35983)
0619 08:51:05.820157 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.820160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.820408 (+ 248us) server_negotiation.cc:407] Connection header received
0619 08:51:05.820991 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.820992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.821004 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.821059 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:05.821842 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.822634 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.824270 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.824603 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.824685 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.827480 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:05.827482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:05.827483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:05.827492 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:05.829610 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.830001 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.830002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.830003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.830070 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.830293 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.830294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.830295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.830456 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:05.830529 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.830770 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.830856 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:51:05.831008 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.820087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:35983 (local address 127.15.208.3:35267)
0619 08:51:05.820270 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:05.820311 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:05.820888 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:05.821153 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:05.821156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:05.821449 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:05.821719 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.821726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.822774 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.822775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.824152 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.824155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.824705 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.824706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.824875 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.825540 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:05.825553 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:05.827358 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:05.829700 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.829705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.829707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.829924 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.830137 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.830138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.830138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.830231 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.830519 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:05.830523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:05.830643 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 08:51:05.830724 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:51:05.830851 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260619 08:51:05.831744 18414 heartbeater.cc:347] Connected to a master server at 127.15.208.62:35983
I20260619 08:51:05.831841 18414 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:05.832038 18414 heartbeater.cc:511] Master 127.15.208.62:35983 requested a full tablet report, sending...
I20260619 08:51:05.832418 17930 ts_manager.cc:194] Registered new tserver with Master: 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081)
I20260619 08:51:05.833007 17930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:35267
I20260619 08:51:05.834431 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 18283
I20260619 08:51:05.834517 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:05.834682 16192 external_mini_cluster.cc:1428] Setting key 7089b1a5c5c0d71d3aa2e6e79e7c364b
I20260619 08:51:05.835480 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:51:05.839146 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.832924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:34295 (local address 127.15.208.62:35983)
0619 08:51:05.832965 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.832967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.832975 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:05.833037 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.833038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.833047 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.833104 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:05.833541 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.834013 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.834991 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.835260 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.835326 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.836367 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:05.836368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:05.836369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:05.836376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:05.837947 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.838346 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.838347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.838347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.838417 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.838631 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.838632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.838633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.838767 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:05.838850 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.839010 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.839086 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:51:05.839226 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.832817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:35983 (local address 127.15.208.3:34295)
0619 08:51:05.832927 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:05.832937 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:05.832989 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:05.833201 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:05.833201 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:05.833409 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:05.833476 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.833478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.834110 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.834111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.834895 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.834897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.835341 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.835342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.835430 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.835865 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:05.835870 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:05.836270 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:05.838026 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.838027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.838028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.838258 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.838493 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:05.838493 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:05.838494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:05.838562 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:05.838844 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:05.838845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:05.838894 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 08:51:05.838971 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:51:05.839086 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":61,"threads_started":1}
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:51:05 dist-test-slave-p0h9 krb5kdc[17886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:05.860253 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.853207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.15.208.62:35983)
0619 08:51:05.853254 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.853255 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.853272 (+ 17us) server_negotiation.cc:407] Connection header received
0619 08:51:05.853335 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.853336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.853344 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.853388 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:05.853954 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.854441 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.855177 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.855359 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.855414 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.857778 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:05.857779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:05.857780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:05.857789 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:05.859239 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.859584 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.859586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.859586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.859647 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:05.859833 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:05.859833 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:05.859834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:05.859945 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:05.860021 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.860121 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.860186 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:05.862097 17930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:05.863862 17930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:05.870563 18433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.866366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.15.208.2:36511)
0619 08:51:05.866519 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.866524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.866648 (+ 124us) server_negotiation.cc:407] Connection header received
0619 08:51:05.866899 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.866902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.866916 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.866989 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:05.867223 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.868283 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.869613 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.870105 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.870208 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.870270 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.870422 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.870462 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260619 08:51:05.870546 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.866256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:36511 (local address 127.0.0.1:50066)
0619 08:51:05.866531 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:05.866575 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:05.866790 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:05.867082 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:05.867083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:05.867086 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:05.867144 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.867146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.868359 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.868360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.869494 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.869496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.870213 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.870214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.870291 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.870304 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 08:51:05.870404 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 08:51:05.870458 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":21,"thread_start_us":90,"threads_started":1}
I20260619 08:51:05.870651 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.866675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53592 (local address 127.15.208.1:35061)
0619 08:51:05.867016 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.867020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.867030 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:05.867066 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.867068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.867082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.867154 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:05.867346 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.868041 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.869537 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.870215 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.870272 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.870319 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.870499 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.870539 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":74,"threads_started":1}
I20260619 08:51:05.870654 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.866462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:35061 (local address 127.0.0.1:53592)
0619 08:51:05.866694 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:05.866703 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:05.866740 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:05.867219 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:05.867219 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:05.867221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:05.867257 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.867258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.868129 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.868130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.869447 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.869450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.870352 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.870353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.870410 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.870413 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:05.870483 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:51:05.870526 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":1,"thread_start_us":74,"threads_started":1}
I20260619 08:51:05.871001 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.866355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52090 (local address 127.15.208.3:40081)
0619 08:51:05.866648 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:05.866653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:05.866665 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:51:05.866764 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:05.866767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:05.866784 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:05.866893 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:05.867097 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.868211 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.869412 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.870158 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.870236 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.870297 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:05.870856 (+ 559us) server_negotiation.cc:299] Negotiation successful
0619 08:51:05.870913 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":84,"threads_started":1}
I20260619 08:51:05.871230 18435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:05.866256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:40081 (local address 127.0.0.1:52090)
0619 08:51:05.866591 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:05.866602 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:05.866659 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:05.866895 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:05.866898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:05.866900 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:05.866991 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.866993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.868331 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.868332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.869327 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:05.869329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:05.870660 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:05.870661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:05.870741 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:05.870745 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:05.871107 (+ 362us) client_negotiation.cc:240] Negotiation successful
0619 08:51:05.871149 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":35,"thread_start_us":63,"threads_started":1}
I20260619 08:51:05.871412 18068 tablet_service.cc:1467] Processing CreateTablet for tablet 4f072d8bec474bfaa58a18dc32a28206 (DEFAULT_TABLE table=test-table [id=20896246e89c4523906024a160c28f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:05.871716 18068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f072d8bec474bfaa58a18dc32a28206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:05.871964 18208 tablet_service.cc:1467] Processing CreateTablet for tablet 4f072d8bec474bfaa58a18dc32a28206 (DEFAULT_TABLE table=test-table [id=20896246e89c4523906024a160c28f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:05.872278 18208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f072d8bec474bfaa58a18dc32a28206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:05.874742 18437 tablet_bootstrap.cc:492] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: Bootstrap starting.
I20260619 08:51:05.875236 18438 tablet_bootstrap.cc:492] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: Bootstrap starting.
I20260619 08:51:05.875385 18437 tablet_bootstrap.cc:654] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:05.875487 18332 tablet_service.cc:1467] Processing CreateTablet for tablet 4f072d8bec474bfaa58a18dc32a28206 (DEFAULT_TABLE table=test-table [id=20896246e89c4523906024a160c28f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:05.875722 18332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f072d8bec474bfaa58a18dc32a28206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:05.875715 18437 log.cc:824] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:05.875823 18438 tablet_bootstrap.cc:654] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:05.876138 18438 log.cc:824] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:05.876504 18437 tablet_bootstrap.cc:492] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: No bootstrap required, opened a new log
I20260619 08:51:05.876583 18437 ts_tablet_manager.cc:1397] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:05.876900 18438 tablet_bootstrap.cc:492] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: No bootstrap required, opened a new log
I20260619 08:51:05.876966 18438 ts_tablet_manager.cc:1397] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:05.878522 18441 tablet_bootstrap.cc:492] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: Bootstrap starting.
I20260619 08:51:05.878574 18437 raft_consensus.cc:348] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } }
I20260619 08:51:05.878764 18437 raft_consensus.cc:374] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:05.878796 18437 raft_consensus.cc:729] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd93b64f7804f70a89e70fe2c90be39, State: Initialized, Role: FOLLOWER
I20260619 08:51:05.878929 18437 consensus_queue.cc:260] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } }
I20260619 08:51:05.878991 18438 raft_consensus.cc:348] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } }
I20260619 08:51:05.879148 18438 raft_consensus.cc:374] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:05.879194 18438 raft_consensus.cc:729] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ad6e382e7e4f9c85e64df427df77f1, State: Initialized, Role: FOLLOWER
I20260619 08:51:05.879204 18437 ts_tablet_manager.cc:1428] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:51:05.879366 18134 heartbeater.cc:503] Master 127.15.208.62:35983 was elected leader, sending a full tablet report...
I20260619 08:51:05.879303 18438 consensus_queue.cc:260] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } }
I20260619 08:51:05.879560 18441 tablet_bootstrap.cc:654] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:05.879590 18438 ts_tablet_manager.cc:1428] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:51:05.879616 18274 heartbeater.cc:503] Master 127.15.208.62:35983 was elected leader, sending a full tablet report...
I20260619 08:51:05.879988 18441 log.cc:824] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:05.880784 18441 tablet_bootstrap.cc:492] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: No bootstrap required, opened a new log
I20260619 08:51:05.880851 18441 ts_tablet_manager.cc:1397] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:51:05.882112 18441 raft_consensus.cc:348] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } }
I20260619 08:51:05.882225 18441 raft_consensus.cc:374] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:05.882246 18441 raft_consensus.cc:729] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831b3bcb69b3453eb9d1ed8566176761, State: Initialized, Role: FOLLOWER
I20260619 08:51:05.882349 18441 consensus_queue.cc:260] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } }
I20260619 08:51:05.882601 18441 ts_tablet_manager.cc:1428] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:51:05.882647 18414 heartbeater.cc:503] Master 127.15.208.62:35983 was elected leader, sending a full tablet report...
W20260619 08:51:05.903662 18275 tablet.cc:2369] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:51:05.995726 18135 tablet.cc:2369] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:51:06.013435 18442 raft_consensus.cc:482] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:06.013612 18442 raft_consensus.cc:504] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } }
I20260619 08:51:06.014007 18442 leader_election.cc:288] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081), e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511)
I20260619 08:51:06.016884 18433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.014276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:57835 (local address 127.15.208.2:36511)
0619 08:51:06.014324 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.014326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.014368 (+ 42us) server_negotiation.cc:407] Connection header received
0619 08:51:06.014450 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.014451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.014459 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.014518 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:06.014708 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.015151 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.016156 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.016519 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.016572 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.016579 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.016759 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.016815 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 08:51:06.016916 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.014318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:46745 (local address 127.15.208.3:40081)
0619 08:51:06.014358 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.014360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.014369 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:06.014447 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.014448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.014463 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.014531 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:06.014671 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.015169 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.016192 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.016529 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.016620 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.016628 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.016815 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.016853 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:51:06.016923 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.014168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:36511 (local address 127.15.208.1:57835)
0619 08:51:06.014298 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.014311 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.014384 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.014527 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.014528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.014530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:06.014625 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.014627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.015227 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.015228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.016073 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.016075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.016598 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.016599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.016666 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.016669 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.016718 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.016814 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":34,"thread_start_us":85,"threads_started":1}
I20260619 08:51:06.016937 18447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.014184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:40081 (local address 127.15.208.1:46745)
0619 08:51:06.014312 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.014323 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.014376 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.014539 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.014540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.014542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:06.014599 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.014601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.015278 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.015278 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.016078 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.016080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.016635 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.016636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.016696 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.016699 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.016807 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.016844 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":38,"thread_start_us":52,"threads_started":1}
I20260619 08:51:06.017210 18228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f072d8bec474bfaa58a18dc32a28206" candidate_uuid: "abd93b64f7804f70a89e70fe2c90be39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" is_pre_election: true
I20260619 08:51:06.017323 18228 raft_consensus.cc:2454] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abd93b64f7804f70a89e70fe2c90be39 in term 0.
I20260619 08:51:06.017534 18019 leader_election.cc:302] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abd93b64f7804f70a89e70fe2c90be39, e6ad6e382e7e4f9c85e64df427df77f1; no voters:
I20260619 08:51:06.017581 18368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f072d8bec474bfaa58a18dc32a28206" candidate_uuid: "abd93b64f7804f70a89e70fe2c90be39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831b3bcb69b3453eb9d1ed8566176761" is_pre_election: true
I20260619 08:51:06.017661 18442 raft_consensus.cc:2781] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:06.017720 18442 raft_consensus.cc:482] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:06.017724 18368 raft_consensus.cc:2454] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abd93b64f7804f70a89e70fe2c90be39 in term 0.
I20260619 08:51:06.017752 18442 raft_consensus.cc:3037] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:06.018415 18442 raft_consensus.cc:504] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } }
I20260619 08:51:06.018567 18442 leader_election.cc:288] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [CANDIDATE]: Term 1 election: Requested vote from peers 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081), e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511)
I20260619 08:51:06.018755 18368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f072d8bec474bfaa58a18dc32a28206" candidate_uuid: "abd93b64f7804f70a89e70fe2c90be39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831b3bcb69b3453eb9d1ed8566176761"
I20260619 08:51:06.018764 18228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f072d8bec474bfaa58a18dc32a28206" candidate_uuid: "abd93b64f7804f70a89e70fe2c90be39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ad6e382e7e4f9c85e64df427df77f1"
I20260619 08:51:06.018829 18368 raft_consensus.cc:3037] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:06.018847 18228 raft_consensus.cc:3037] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:06.019481 18368 raft_consensus.cc:2454] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abd93b64f7804f70a89e70fe2c90be39 in term 1.
I20260619 08:51:06.019611 18228 raft_consensus.cc:2454] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abd93b64f7804f70a89e70fe2c90be39 in term 1.
I20260619 08:51:06.019663 18021 leader_election.cc:302] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 831b3bcb69b3453eb9d1ed8566176761, abd93b64f7804f70a89e70fe2c90be39; no voters:
I20260619 08:51:06.019766 18442 raft_consensus.cc:2781] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:06.019886 18442 raft_consensus.cc:686] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 1 LEADER]: Becoming Leader. State: Replica: abd93b64f7804f70a89e70fe2c90be39, State: Running, Role: LEADER
I20260619 08:51:06.019984 18442 consensus_queue.cc:237] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } }
I20260619 08:51:06.020757 17930 catalog_manager.cc:5495] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 reported cstate change: term changed from 0 to 1, leader changed from <none> to abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1). New cstate: current_term: 1 leader_uuid: "abd93b64f7804f70a89e70fe2c90be39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } health_report { overall_health: HEALTHY } } }
Jun 19 08:51:06 dist-test-slave-p0h9 krb5kdc[17886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:06.068519 18455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.061481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:35983 (local address 127.0.0.1:49176)
0619 08:51:06.061688 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.061698 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.061756 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.061908 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.061909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.062218 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:06.062275 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.062277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.063008 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.063009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.063632 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.063634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.064023 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.064024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.064067 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.064684 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:06.064688 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:06.066118 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:06.067595 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:06.067601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:06.067603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:06.067789 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:06.067968 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:06.067968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:06.067969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:06.068022 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:06.068228 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:06.068233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:06.068271 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.068324 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.068414 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":108,"threads_started":1}
I20260619 08:51:06.068531 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.061569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49176 (local address 127.15.208.62:35983)
0619 08:51:06.061644 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.061646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.061806 (+ 160us) server_negotiation.cc:407] Connection header received
0619 08:51:06.061833 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.061833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.061845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.061894 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:06.062342 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.062894 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.063723 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.063904 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.064033 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.066197 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:06.066199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:06.066199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:06.066205 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:06.067517 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:06.067854 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:06.067854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:06.067855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:06.067904 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:06.068080 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:06.068080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:06.068081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:06.068174 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:06.068223 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.068352 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.068459 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":48}
I20260619 08:51:06.070627 17930 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:49176:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 08:51:06.070852 17930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:06.072255 18068 tablet_service.cc:1467] Processing CreateTablet for tablet 1b55785a13a54ea4936b5fcb3657f16e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=43dd87b2c71d4305891a220ce3b983e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:51:06.072269 18332 tablet_service.cc:1467] Processing CreateTablet for tablet 1b55785a13a54ea4936b5fcb3657f16e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=43dd87b2c71d4305891a220ce3b983e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:51:06.072436 18332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b55785a13a54ea4936b5fcb3657f16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:06.072474 18068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b55785a13a54ea4936b5fcb3657f16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:06.073776 18437 tablet_bootstrap.cc:492] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: Bootstrap starting.
I20260619 08:51:06.073834 18441 tablet_bootstrap.cc:492] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: Bootstrap starting.
I20260619 08:51:06.073973 18208 tablet_service.cc:1467] Processing CreateTablet for tablet 1b55785a13a54ea4936b5fcb3657f16e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=43dd87b2c71d4305891a220ce3b983e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:51:06.074115 18208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b55785a13a54ea4936b5fcb3657f16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:06.074210 18437 tablet_bootstrap.cc:654] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:06.074276 18441 tablet_bootstrap.cc:654] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:06.074870 18441 tablet_bootstrap.cc:492] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: No bootstrap required, opened a new log
I20260619 08:51:06.074910 18441 ts_tablet_manager.cc:1397] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:06.075067 18441 raft_consensus.cc:348] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.075126 18441 raft_consensus.cc:374] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:06.075140 18441 raft_consensus.cc:729] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831b3bcb69b3453eb9d1ed8566176761, State: Initialized, Role: FOLLOWER
I20260619 08:51:06.075194 18441 consensus_queue.cc:260] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.075289 18441 ts_tablet_manager.cc:1428] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:06.075340 18445 tablet_replica.cc:439] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } }
I20260619 08:51:06.075425 18445 tablet_replica.cc:442] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:06.075649 18438 tablet_bootstrap.cc:492] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: Bootstrap starting.
I20260619 08:51:06.076372 18438 tablet_bootstrap.cc:654] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:06.076838 18437 tablet_bootstrap.cc:492] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: No bootstrap required, opened a new log
I20260619 08:51:06.076902 18437 ts_tablet_manager.cc:1397] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 08:51:06.077016 18438 tablet_bootstrap.cc:492] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: No bootstrap required, opened a new log
I20260619 08:51:06.077060 18438 ts_tablet_manager.cc:1397] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:06.077078 18437 raft_consensus.cc:348] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.077152 18437 raft_consensus.cc:374] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:06.077183 18437 raft_consensus.cc:729] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd93b64f7804f70a89e70fe2c90be39, State: Initialized, Role: FOLLOWER
I20260619 08:51:06.077234 18437 consensus_queue.cc:260] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.077220 18438 raft_consensus.cc:348] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.077278 18438 raft_consensus.cc:374] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:06.077308 18438 raft_consensus.cc:729] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ad6e382e7e4f9c85e64df427df77f1, State: Initialized, Role: FOLLOWER
I20260619 08:51:06.077322 18437 ts_tablet_manager.cc:1428] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:06.077359 18438 consensus_queue.cc:260] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.077443 18438 ts_tablet_manager.cc:1428] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:06.077448 18442 tablet_replica.cc:439] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } }
I20260619 08:51:06.077534 18442 tablet_replica.cc:442] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:06.077585 18443 tablet_replica.cc:439] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } }
I20260619 08:51:06.077659 18443 tablet_replica.cc:442] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 08:51:06.078739 18415 tablet.cc:2369] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:51:06.214123 18445 raft_consensus.cc:482] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:06.214241 18445 raft_consensus.cc:504] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.214672 18445 leader_election.cc:288] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061), e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511)
I20260619 08:51:06.217653 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.214943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:57139 (local address 127.15.208.1:35061)
0619 08:51:06.214999 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.215001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.215012 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:51:06.215082 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.215084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.215093 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.215178 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:06.215350 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.215861 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.216917 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.217277 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.217363 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.217371 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.217537 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.217572 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:51:06.217653 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.214892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:35061 (local address 127.15.208.3:57139)
0619 08:51:06.214950 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.214961 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.215010 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.215183 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.215184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.215186 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:06.215268 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.215270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.215962 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.215963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.216810 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.216812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.217362 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.217363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.217456 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.217459 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.217531 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.217572 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":29}
I20260619 08:51:06.217768 18433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.214987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:50427 (local address 127.15.208.2:36511)
0619 08:51:06.215028 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.215030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.215162 (+ 132us) server_negotiation.cc:407] Connection header received
0619 08:51:06.215250 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.215251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.215261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.215332 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:06.215503 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.215982 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.216998 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.217356 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.217441 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.217451 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.217626 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.217665 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:51:06.217983 18459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.214926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:36511 (local address 127.15.208.3:50427)
0619 08:51:06.215082 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.215098 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.215177 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.215349 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.215349 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.215351 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:06.215427 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.215429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.216085 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.216086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.216917 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.216919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.217458 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.217459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.217536 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.217538 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.217618 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.217657 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260619 08:51:06.217990 18088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b55785a13a54ea4936b5fcb3657f16e" candidate_uuid: "831b3bcb69b3453eb9d1ed8566176761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abd93b64f7804f70a89e70fe2c90be39" is_pre_election: true
I20260619 08:51:06.218120 18088 raft_consensus.cc:2454] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831b3bcb69b3453eb9d1ed8566176761 in term 0.
I20260619 08:51:06.218266 18228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b55785a13a54ea4936b5fcb3657f16e" candidate_uuid: "831b3bcb69b3453eb9d1ed8566176761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" is_pre_election: true
I20260619 08:51:06.218331 18228 raft_consensus.cc:2454] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831b3bcb69b3453eb9d1ed8566176761 in term 0.
I20260619 08:51:06.218314 18301 leader_election.cc:302] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 831b3bcb69b3453eb9d1ed8566176761, abd93b64f7804f70a89e70fe2c90be39; no voters:
I20260619 08:51:06.218533 18445 raft_consensus.cc:2781] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:06.218576 18445 raft_consensus.cc:482] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:06.218591 18445 raft_consensus.cc:3037] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:06.219252 18445 raft_consensus.cc:504] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.219389 18445 leader_election.cc:288] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [CANDIDATE]: Term 1 election: Requested vote from peers abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061), e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511)
I20260619 08:51:06.219539 18088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b55785a13a54ea4936b5fcb3657f16e" candidate_uuid: "831b3bcb69b3453eb9d1ed8566176761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abd93b64f7804f70a89e70fe2c90be39"
I20260619 08:51:06.219638 18088 raft_consensus.cc:3037] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:06.219592 18228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b55785a13a54ea4936b5fcb3657f16e" candidate_uuid: "831b3bcb69b3453eb9d1ed8566176761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ad6e382e7e4f9c85e64df427df77f1"
I20260619 08:51:06.219707 18228 raft_consensus.cc:3037] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:06.220320 18228 raft_consensus.cc:2454] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 831b3bcb69b3453eb9d1ed8566176761 in term 1.
I20260619 08:51:06.220342 18088 raft_consensus.cc:2454] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 831b3bcb69b3453eb9d1ed8566176761 in term 1.
I20260619 08:51:06.220489 18301 leader_election.cc:302] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 831b3bcb69b3453eb9d1ed8566176761, abd93b64f7804f70a89e70fe2c90be39; no voters:
I20260619 08:51:06.220573 18445 raft_consensus.cc:2781] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:06.220749 18445 raft_consensus.cc:686] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [term 1 LEADER]: Becoming Leader. State: Replica: 831b3bcb69b3453eb9d1ed8566176761, State: Running, Role: LEADER
I20260619 08:51:06.220867 18445 consensus_queue.cc:237] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.221328 18460 tablet_replica.cc:439] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: TxnStatusTablet state changed. Reason: New leader 831b3bcb69b3453eb9d1ed8566176761. Latest consensus state: current_term: 1 leader_uuid: "831b3bcb69b3453eb9d1ed8566176761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } }
I20260619 08:51:06.221414 18460 tablet_replica.cc:442] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:51:06.221592 18462 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 08:51:06.221760 18462 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 08:51:06.221750 17930 catalog_manager.cc:5495] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 reported cstate change: term changed from 0 to 1, leader changed from <none> to 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3). New cstate: current_term: 1 leader_uuid: "831b3bcb69b3453eb9d1ed8566176761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } health_report { overall_health: HEALTHY } } }
I20260619 08:51:06.263309 18455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.260347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:40081 (local address 127.0.0.1:52094)
0619 08:51:06.260457 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.260469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.260536 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.260772 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.260773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.260775 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:06.260833 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.260841 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.261754 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.261755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.262316 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.262318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.262785 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.262786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.262832 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.262848 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.263135 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.263141 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.263163 (+ 22us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.263240 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260619 08:51:06.263338 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.260420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52094 (local address 127.15.208.3:40081)
0619 08:51:06.260471 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.260473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.260517 (+ 44us) server_negotiation.cc:407] Connection header received
0619 08:51:06.260627 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.260628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.260658 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.260743 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:06.260910 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.261672 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.262397 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.262708 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.262778 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.262936 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.263064 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.263149 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.263215 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.263237 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:51:06.265224 18228 raft_consensus.cc:1261] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Refusing update from remote peer 831b3bcb69b3453eb9d1ed8566176761: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:06.265223 18088 raft_consensus.cc:1261] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Refusing update from remote peer 831b3bcb69b3453eb9d1ed8566176761: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:06.265547 18445 consensus_queue.cc:1035] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:06.265667 18460 consensus_queue.cc:1035] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:06.266774 18442 tablet_replica.cc:439] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: TxnStatusTablet state changed. Reason: New leader 831b3bcb69b3453eb9d1ed8566176761. Latest consensus state: current_term: 1 leader_uuid: "831b3bcb69b3453eb9d1ed8566176761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } }
I20260619 08:51:06.266808 18443 tablet_replica.cc:439] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: TxnStatusTablet state changed. Reason: New leader 831b3bcb69b3453eb9d1ed8566176761. Latest consensus state: current_term: 1 leader_uuid: "831b3bcb69b3453eb9d1ed8566176761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } }
I20260619 08:51:06.266870 18442 tablet_replica.cc:442] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:06.266897 18443 tablet_replica.cc:442] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:06.268239 18443 tablet_replica.cc:439] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "831b3bcb69b3453eb9d1ed8566176761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } }
I20260619 08:51:06.268323 18443 tablet_replica.cc:442] T 1b55785a13a54ea4936b5fcb3657f16e P e6ad6e382e7e4f9c85e64df427df77f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:06.268359 18467 mvcc.cc:205] Tried to move back new op lower bound from 7298494735419629568 to 7298494735241244672. Current Snapshot: MvccSnapshot[applied={T|T < 7298494735419629568}]
I20260619 08:51:06.268913 18442 tablet_replica.cc:439] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "831b3bcb69b3453eb9d1ed8566176761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } }
I20260619 08:51:06.268992 18442 tablet_replica.cc:442] T 1b55785a13a54ea4936b5fcb3657f16e P abd93b64f7804f70a89e70fe2c90be39: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:06.269294 18460 tablet_replica.cc:439] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "831b3bcb69b3453eb9d1ed8566176761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } }
I20260619 08:51:06.269403 18460 tablet_replica.cc:442] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:51:06.269527 18445 tablet_replica.cc:439] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "831b3bcb69b3453eb9d1ed8566176761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } } }
I20260619 08:51:06.269591 18445 tablet_replica.cc:442] T 1b55785a13a54ea4936b5fcb3657f16e P 831b3bcb69b3453eb9d1ed8566176761: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:51:06.269709 18465 mvcc.cc:205] Tried to move back new op lower bound from 7298494735419629568 to 7298494735241244672. Current Snapshot: MvccSnapshot[applied={T|T < 7298494735419629568}]
I20260619 08:51:06.274672 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.271542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53594 (local address 127.15.208.1:35061)
0619 08:51:06.271582 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.271584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.271593 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:06.271659 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.271660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.271678 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.271758 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:06.271879 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.272652 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.273789 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.274089 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.274163 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.274334 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.274449 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.274526 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.274572 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.274594 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:51:06.279575 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.277133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:50717 (local address 127.15.208.3:40081)
0619 08:51:06.277193 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.277195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.277235 (+ 40us) server_negotiation.cc:407] Connection header received
0619 08:51:06.277298 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.277299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.277306 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.277352 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:06.277486 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.277929 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.278937 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.279107 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.279156 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.279350 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.279373 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.279410 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.279490 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.279505 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:51:06.279628 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.276985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:40081 (local address 127.15.208.1:50717)
0619 08:51:06.277169 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.277181 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.277240 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.277371 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.277372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.277374 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:06.277427 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.277428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.277990 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.277990 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.278856 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.278858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.279177 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.279178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.279244 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.279259 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.279433 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.279435 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.279485 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.279537 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260619 08:51:06.282182 18228 raft_consensus.cc:1261] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Refusing update from remote peer abd93b64f7804f70a89e70fe2c90be39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:06.282312 18368 raft_consensus.cc:1261] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Refusing update from remote peer abd93b64f7804f70a89e70fe2c90be39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:06.282495 18442 consensus_queue.cc:1035] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:06.282655 18448 consensus_queue.cc:1035] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:06.291853 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.288617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:33133 (local address 127.15.208.1:35061)
0619 08:51:06.288665 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.288666 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.288674 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:06.288696 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.288696 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.288706 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.288794 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:06.288984 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.289687 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.291220 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.291399 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.291453 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.291636 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.291660 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.291697 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.291752 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.291773 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 08:51:06.291870 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.288507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:35061 (local address 127.15.208.3:33133)
0619 08:51:06.288574 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.288588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.288629 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.288842 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.288843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.288845 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:06.288908 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.288910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.290210 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.290212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.291139 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.291142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.291470 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.291471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.291537 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.291552 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.291699 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.291700 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.291749 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.291791 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 08:51:06.301371 17930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:06.301492 17930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:06.302279 17930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285: Sending DeleteTablet for 3 replicas of tablet 4f072d8bec474bfaa58a18dc32a28206
I20260619 08:51:06.302580 18208 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f072d8bec474bfaa58a18dc32a28206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:06 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:50066
I20260619 08:51:06.302707 18332 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f072d8bec474bfaa58a18dc32a28206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:06 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:52090
I20260619 08:51:06.302877 17930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 08:51:06.302946 18486 tablet_replica.cc:330] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: stopping tablet replica
I20260619 08:51:06.302959 18068 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f072d8bec474bfaa58a18dc32a28206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:06 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53592
W20260619 08:51:06.303025 17930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:06.303269 18486 raft_consensus.cc:2229] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:06.303408 18486 raft_consensus.cc:2258] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:06.303941 18486 ts_tablet_manager.cc:1905] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:06.304138 18208 tablet_service.cc:1467] Processing CreateTablet for tablet 1e3a92401d404f6486174d44e8a0b40b (DEFAULT_TABLE table=test-table [id=ab581dd3d4c341c686dbc4a756a7c622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:06.304255 18208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e3a92401d404f6486174d44e8a0b40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:06.305649 18438 tablet_bootstrap.cc:492] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1: Bootstrap starting.
I20260619 08:51:06.305924 18486 ts_tablet_manager.cc:1918] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:06.306052 18486 log.cc:1196] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/4f072d8bec474bfaa58a18dc32a28206
I20260619 08:51:06.306069 18438 tablet_bootstrap.cc:654] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:06.306454 18486 ts_tablet_manager.cc:1939] T 4f072d8bec474bfaa58a18dc32a28206 P 831b3bcb69b3453eb9d1ed8566176761: Deleting consensus metadata
I20260619 08:51:06.306720 18068 tablet_service.cc:1467] Processing CreateTablet for tablet 1e3a92401d404f6486174d44e8a0b40b (DEFAULT_TABLE table=test-table [id=ab581dd3d4c341c686dbc4a756a7c622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:06.306836 18068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e3a92401d404f6486174d44e8a0b40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:06.307034 17917 catalog_manager.cc:4841] TS 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081): tablet 4f072d8bec474bfaa58a18dc32a28206 (table test-table [id=20896246e89c4523906024a160c28f94]) successfully deleted
I20260619 08:51:06.307891 18437 tablet_bootstrap.cc:492] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39: Bootstrap starting.
I20260619 08:51:06.307928 18332 tablet_service.cc:1467] Processing CreateTablet for tablet 1e3a92401d404f6486174d44e8a0b40b (DEFAULT_TABLE table=test-table [id=ab581dd3d4c341c686dbc4a756a7c622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:06.308017 18332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e3a92401d404f6486174d44e8a0b40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:06.308295 18437 tablet_bootstrap.cc:654] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:06.309000 18437 tablet_bootstrap.cc:492] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39: No bootstrap required, opened a new log
I20260619 08:51:06.309044 18437 ts_tablet_manager.cc:1397] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:06.309185 18437 raft_consensus.cc:348] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.309273 18437 raft_consensus.cc:374] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:06.309299 18437 raft_consensus.cc:729] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd93b64f7804f70a89e70fe2c90be39, State: Initialized, Role: FOLLOWER
I20260619 08:51:06.309314 18441 tablet_bootstrap.cc:492] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761: Bootstrap starting.
I20260619 08:51:06.309341 18437 consensus_queue.cc:260] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.309427 18437 ts_tablet_manager.cc:1428] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:06.309701 18441 tablet_bootstrap.cc:654] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:06.311012 18438 tablet_bootstrap.cc:492] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1: No bootstrap required, opened a new log
I20260619 08:51:06.311008 18441 tablet_bootstrap.cc:492] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761: No bootstrap required, opened a new log
I20260619 08:51:06.311054 18441 ts_tablet_manager.cc:1397] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 08:51:06.311064 18438 ts_tablet_manager.cc:1397] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 08:51:06.311193 18441 raft_consensus.cc:348] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.311209 18438 raft_consensus.cc:348] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.311252 18441 raft_consensus.cc:374] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:06.311272 18438 raft_consensus.cc:374] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:06.311272 18441 raft_consensus.cc:729] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831b3bcb69b3453eb9d1ed8566176761, State: Initialized, Role: FOLLOWER
I20260619 08:51:06.311292 18438 raft_consensus.cc:729] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ad6e382e7e4f9c85e64df427df77f1, State: Initialized, Role: FOLLOWER
I20260619 08:51:06.311314 18441 consensus_queue.cc:260] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.311331 18438 consensus_queue.cc:260] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.311396 18441 ts_tablet_manager.cc:1428] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:06.311415 18438 ts_tablet_manager.cc:1428] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:06.312811 18487 tablet_replica.cc:330] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: stopping tablet replica
I20260619 08:51:06.312932 18487 raft_consensus.cc:2229] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:06.313041 18487 raft_consensus.cc:2258] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:06.313426 18487 ts_tablet_manager.cc:1905] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:06.314953 18487 ts_tablet_manager.cc:1918] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:06.315042 18487 log.cc:1196] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/4f072d8bec474bfaa58a18dc32a28206
I20260619 08:51:06.315289 18487 ts_tablet_manager.cc:1939] T 4f072d8bec474bfaa58a18dc32a28206 P abd93b64f7804f70a89e70fe2c90be39: Deleting consensus metadata
I20260619 08:51:06.315675 17917 catalog_manager.cc:4841] TS abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061): tablet 4f072d8bec474bfaa58a18dc32a28206 (table test-table [id=20896246e89c4523906024a160c28f94]) successfully deleted
I20260619 08:51:06.316818 18485 tablet_replica.cc:330] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: stopping tablet replica
I20260619 08:51:06.316946 18485 raft_consensus.cc:2229] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:06.317014 18485 raft_consensus.cc:2258] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:06.317435 18485 ts_tablet_manager.cc:1905] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:06.318677 18485 ts_tablet_manager.cc:1918] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:06.318764 18485 log.cc:1196] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/4f072d8bec474bfaa58a18dc32a28206
I20260619 08:51:06.319043 18485 ts_tablet_manager.cc:1939] T 4f072d8bec474bfaa58a18dc32a28206 P e6ad6e382e7e4f9c85e64df427df77f1: Deleting consensus metadata
I20260619 08:51:06.319437 17915 catalog_manager.cc:4841] TS e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511): tablet 4f072d8bec474bfaa58a18dc32a28206 (table test-table [id=20896246e89c4523906024a160c28f94]) successfully deleted
I20260619 08:51:06.422915 18443 raft_consensus.cc:482] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:06.423040 18443 raft_consensus.cc:504] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.423404 18443 leader_election.cc:288] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061), 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081)
I20260619 08:51:06.427475 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.423934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:54625 (local address 127.15.208.3:40081)
0619 08:51:06.423979 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.423980 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.423988 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:06.424017 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.424018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.424030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.424117 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:06.424381 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.425106 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.426523 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.427075 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.427149 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.427158 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.427367 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.427404 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:06.427497 18492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.423815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:40081 (local address 127.15.208.2:54625)
0619 08:51:06.423892 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.423901 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.423940 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.424158 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.424158 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.424162 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:06.424281 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.424283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.425222 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.425223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.426432 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.426434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.427206 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.427207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.427288 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.427291 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.427361 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.427401 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":44,"thread_start_us":35,"threads_started":1}
I20260619 08:51:06.427502 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.423746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:50899 (local address 127.15.208.1:35061)
0619 08:51:06.423823 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.423826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.423898 (+ 72us) server_negotiation.cc:407] Connection header received
0619 08:51:06.424014 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.424015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.424034 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.424066 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:06.424383 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.425228 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.426577 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.427081 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.427149 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.427158 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.427394 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.427435 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:51:06.427767 18368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3a92401d404f6486174d44e8a0b40b" candidate_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831b3bcb69b3453eb9d1ed8566176761" is_pre_election: true
I20260619 08:51:06.427860 18368 raft_consensus.cc:2454] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6ad6e382e7e4f9c85e64df427df77f1 in term 0.
I20260619 08:51:06.427954 18491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.423623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:35061 (local address 127.15.208.2:50899)
0619 08:51:06.423825 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.423838 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.423930 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.424156 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.424157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.424159 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:06.424270 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.424272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.425332 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.425334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.426452 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.426455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.427205 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.427206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.427291 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.427296 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.427826 (+ 530us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.427872 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":10,"thread_start_us":64,"threads_started":1}
I20260619 08:51:06.428027 18161 leader_election.cc:302] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 831b3bcb69b3453eb9d1ed8566176761, e6ad6e382e7e4f9c85e64df427df77f1; no voters:
I20260619 08:51:06.428169 18443 raft_consensus.cc:2781] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:06.428216 18443 raft_consensus.cc:482] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:06.428246 18443 raft_consensus.cc:3037] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:06.428314 18088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3a92401d404f6486174d44e8a0b40b" candidate_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abd93b64f7804f70a89e70fe2c90be39" is_pre_election: true
I20260619 08:51:06.428401 18088 raft_consensus.cc:2454] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6ad6e382e7e4f9c85e64df427df77f1 in term 0.
I20260619 08:51:06.429240 18443 raft_consensus.cc:504] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.429383 18443 leader_election.cc:288] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [CANDIDATE]: Term 1 election: Requested vote from peers abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061), 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081)
I20260619 08:51:06.429572 18088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3a92401d404f6486174d44e8a0b40b" candidate_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abd93b64f7804f70a89e70fe2c90be39"
I20260619 08:51:06.429590 18368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3a92401d404f6486174d44e8a0b40b" candidate_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831b3bcb69b3453eb9d1ed8566176761"
I20260619 08:51:06.429672 18368 raft_consensus.cc:3037] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:06.429670 18088 raft_consensus.cc:3037] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:06.430280 18088 raft_consensus.cc:2454] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ad6e382e7e4f9c85e64df427df77f1 in term 1.
I20260619 08:51:06.430442 18161 leader_election.cc:302] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abd93b64f7804f70a89e70fe2c90be39, e6ad6e382e7e4f9c85e64df427df77f1; no voters:
I20260619 08:51:06.430557 18443 raft_consensus.cc:2781] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:06.430691 18443 raft_consensus.cc:686] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 LEADER]: Becoming Leader. State: Replica: e6ad6e382e7e4f9c85e64df427df77f1, State: Running, Role: LEADER
I20260619 08:51:06.430732 18368 raft_consensus.cc:2454] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ad6e382e7e4f9c85e64df427df77f1 in term 1.
I20260619 08:51:06.430761 18443 consensus_queue.cc:237] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:06.431548 17931 catalog_manager.cc:5495] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2). New cstate: current_term: 1 leader_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:06.461308 18433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.458895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50070 (local address 127.15.208.2:36511)
0619 08:51:06.458939 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.458941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.458950 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:06.458989 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.458990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.459011 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.459106 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:06.459217 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.459718 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.460455 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.460664 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.460762 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.460932 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.461068 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.461166 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.461204 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.461225 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:06.466178 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.463914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:56377 (local address 127.15.208.3:40081)
0619 08:51:06.463996 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.463998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.464048 (+ 50us) server_negotiation.cc:407] Connection header received
0619 08:51:06.464107 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.464108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.464119 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.464185 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:06.464331 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.464872 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.465629 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.465809 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.465862 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.466005 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.466035 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.466072 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.466106 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.466123 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 08:51:06.466212 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.463844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:40081 (local address 127.15.208.2:56377)
0619 08:51:06.463981 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.463991 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.464048 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.464188 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.464188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.464190 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:06.464243 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.464244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.464969 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.464970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.465536 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.465538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.465874 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.465875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.465924 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.465936 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.466072 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.466073 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.466103 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.466132 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260619 08:51:06.468236 18368 raft_consensus.cc:1261] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Refusing update from remote peer e6ad6e382e7e4f9c85e64df427df77f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:06.468236 18088 raft_consensus.cc:1261] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Refusing update from remote peer e6ad6e382e7e4f9c85e64df427df77f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:06.468773 18493 consensus_queue.cc:1035] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:06.468959 18443 consensus_queue.cc:1035] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:06.470132 18466 mvcc.cc:205] Tried to move back new op lower bound from 7298494736252055552 to 7298494736100921344. Current Snapshot: MvccSnapshot[applied={T|T < 7298494736252055552}]
I20260619 08:51:06.475638 18433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.473454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:36429 (local address 127.15.208.2:36511)
0619 08:51:06.473497 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:06.473499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:06.473507 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:06.473562 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:06.473562 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:06.473572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:06.473626 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:06.473722 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.474181 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.474959 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.475216 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.475278 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.475408 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.475440 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.475490 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:06.475551 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 08:51:06.475570 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:51:06.475657 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:06.473414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:36511 (local address 127.15.208.3:36429)
0619 08:51:06.473461 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:06.473472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:06.473510 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:06.473630 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:06.473630 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:06.473631 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:06.473671 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.473673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.474266 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.474267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.474856 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:06.474857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:06.475276 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:06.475276 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:06.475332 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:06.475337 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:06.475495 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:06.475497 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:06.475537 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 08:51:06.475590 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:51:06.476024 18467 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 08:51:07.994266 17931 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:07.994364 17931 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:07.995177 17931 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285: Sending DeleteTablet for 3 replicas of tablet 1e3a92401d404f6486174d44e8a0b40b
I20260619 08:51:07.995434 18208 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e3a92401d404f6486174d44e8a0b40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:07 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:50066
I20260619 08:51:07.995482 18068 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e3a92401d404f6486174d44e8a0b40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:07 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53592
I20260619 08:51:07.995510 18332 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e3a92401d404f6486174d44e8a0b40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:07 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:52090
I20260619 08:51:07.995684 18525 tablet_replica.cc:330] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761: stopping tablet replica
I20260619 08:51:07.995744 18525 raft_consensus.cc:2229] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:07.995787 18525 raft_consensus.cc:2258] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:07.995905 18523 tablet_replica.cc:330] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1: stopping tablet replica
I20260619 08:51:07.995985 18523 raft_consensus.cc:2229] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:07.996078 18523 raft_consensus.cc:2258] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:07.996290 18525 ts_tablet_manager.cc:1905] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:07.996825 18524 tablet_replica.cc:330] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39: stopping tablet replica
I20260619 08:51:07.996902 18524 raft_consensus.cc:2229] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:07.996948 18524 raft_consensus.cc:2258] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:07.997219 18523 ts_tablet_manager.cc:1905] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:07.997884 18524 ts_tablet_manager.cc:1905] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:07.998718 18523 ts_tablet_manager.cc:1918] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:51:07.998793 18523 log.cc:1196] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/1e3a92401d404f6486174d44e8a0b40b
I20260619 08:51:07.999044 18523 ts_tablet_manager.cc:1939] T 1e3a92401d404f6486174d44e8a0b40b P e6ad6e382e7e4f9c85e64df427df77f1: Deleting consensus metadata
I20260619 08:51:07.999393 17915 catalog_manager.cc:4841] TS e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511): tablet 1e3a92401d404f6486174d44e8a0b40b (table test-table [id=ab581dd3d4c341c686dbc4a756a7c622]) successfully deleted
I20260619 08:51:08.000159 18524 ts_tablet_manager.cc:1918] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:51:08.000211 18524 log.cc:1196] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/1e3a92401d404f6486174d44e8a0b40b
I20260619 08:51:08.000428 18524 ts_tablet_manager.cc:1939] T 1e3a92401d404f6486174d44e8a0b40b P abd93b64f7804f70a89e70fe2c90be39: Deleting consensus metadata
I20260619 08:51:08.000626 18525 ts_tablet_manager.cc:1918] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:51:08.000674 18525 log.cc:1196] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/1e3a92401d404f6486174d44e8a0b40b
I20260619 08:51:08.000898 17917 catalog_manager.cc:4841] TS abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061): tablet 1e3a92401d404f6486174d44e8a0b40b (table test-table [id=ab581dd3d4c341c686dbc4a756a7c622]) successfully deleted
I20260619 08:51:08.000921 18525 ts_tablet_manager.cc:1939] T 1e3a92401d404f6486174d44e8a0b40b P 831b3bcb69b3453eb9d1ed8566176761: Deleting consensus metadata
I20260619 08:51:08.001255 17917 catalog_manager.cc:4841] TS 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081): tablet 1e3a92401d404f6486174d44e8a0b40b (table test-table [id=ab581dd3d4c341c686dbc4a756a7c622]) successfully deleted
Jun 19 08:51:08 dist-test-slave-p0h9 krb5kdc[17886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859068, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:51:08 dist-test-slave-p0h9 krb5kdc[17886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859068, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:08.033303 18533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:08.014122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.15.208.62:35983)
0619 08:51:08.014251 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:08.014253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:08.014263 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:08.014316 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:08.014317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:08.014327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:08.014389 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:08.014973 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:08.015480 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:08.016176 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:08.016363 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:08.016415 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:08.030688 (+ 14273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:08.030690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:08.030690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:08.030699 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:08.032130 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:08.032571 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:08.032571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:08.032572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:08.032624 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:08.032839 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:08.032840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:08.032840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:08.032954 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:08.033031 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:08.033159 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 08:51:08.033228 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260619 08:51:08.034807 17931 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:08.035005 17931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:08.036335 18068 tablet_service.cc:1467] Processing CreateTablet for tablet dc9fa59f48134fea9484373faf2e8ee4 (DEFAULT_TABLE table=test-table [id=19db1d28632648879b0050a33f3871d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:08.036360 18332 tablet_service.cc:1467] Processing CreateTablet for tablet dc9fa59f48134fea9484373faf2e8ee4 (DEFAULT_TABLE table=test-table [id=19db1d28632648879b0050a33f3871d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:08.036511 18068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9fa59f48134fea9484373faf2e8ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:08.036513 18332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9fa59f48134fea9484373faf2e8ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:08.037688 18534 tablet_bootstrap.cc:492] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39: Bootstrap starting.
I20260619 08:51:08.037676 18208 tablet_service.cc:1467] Processing CreateTablet for tablet dc9fa59f48134fea9484373faf2e8ee4 (DEFAULT_TABLE table=test-table [id=19db1d28632648879b0050a33f3871d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:08.037719 18535 tablet_bootstrap.cc:492] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761: Bootstrap starting.
I20260619 08:51:08.037793 18208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9fa59f48134fea9484373faf2e8ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:08.038071 18535 tablet_bootstrap.cc:654] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:08.038079 18534 tablet_bootstrap.cc:654] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:08.038635 18535 tablet_bootstrap.cc:492] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761: No bootstrap required, opened a new log
I20260619 08:51:08.038697 18535 ts_tablet_manager.cc:1397] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:08.038815 18535 raft_consensus.cc:348] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.038889 18535 raft_consensus.cc:374] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:08.038904 18535 raft_consensus.cc:729] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831b3bcb69b3453eb9d1ed8566176761, State: Initialized, Role: FOLLOWER
I20260619 08:51:08.038959 18535 consensus_queue.cc:260] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.039060 18538 tablet_bootstrap.cc:492] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1: Bootstrap starting.
I20260619 08:51:08.039441 18538 tablet_bootstrap.cc:654] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:08.039623 18534 tablet_bootstrap.cc:492] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39: No bootstrap required, opened a new log
I20260619 08:51:08.039675 18534 ts_tablet_manager.cc:1397] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:08.039681 18535 ts_tablet_manager.cc:1428] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:08.039798 18534 raft_consensus.cc:348] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.039884 18534 raft_consensus.cc:374] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:08.039909 18534 raft_consensus.cc:729] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd93b64f7804f70a89e70fe2c90be39, State: Initialized, Role: FOLLOWER
I20260619 08:51:08.039958 18534 consensus_queue.cc:260] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.040113 18534 ts_tablet_manager.cc:1428] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:08.040154 18538 tablet_bootstrap.cc:492] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1: No bootstrap required, opened a new log
I20260619 08:51:08.040210 18538 ts_tablet_manager.cc:1397] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:08.040315 18538 raft_consensus.cc:348] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.040383 18538 raft_consensus.cc:374] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:08.040403 18538 raft_consensus.cc:729] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ad6e382e7e4f9c85e64df427df77f1, State: Initialized, Role: FOLLOWER
I20260619 08:51:08.040464 18538 consensus_queue.cc:260] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.040591 18538 ts_tablet_manager.cc:1428] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:08.306886 18471 raft_consensus.cc:482] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:08.306989 18471 raft_consensus.cc:504] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.307161 18471 leader_election.cc:288] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061), e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511)
I20260619 08:51:08.307438 18228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9fa59f48134fea9484373faf2e8ee4" candidate_uuid: "831b3bcb69b3453eb9d1ed8566176761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" is_pre_election: true
I20260619 08:51:08.307442 18088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9fa59f48134fea9484373faf2e8ee4" candidate_uuid: "831b3bcb69b3453eb9d1ed8566176761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abd93b64f7804f70a89e70fe2c90be39" is_pre_election: true
I20260619 08:51:08.307542 18228 raft_consensus.cc:2454] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831b3bcb69b3453eb9d1ed8566176761 in term 0.
I20260619 08:51:08.307586 18088 raft_consensus.cc:2454] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 831b3bcb69b3453eb9d1ed8566176761 in term 0.
I20260619 08:51:08.307761 18299 leader_election.cc:302] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 831b3bcb69b3453eb9d1ed8566176761, e6ad6e382e7e4f9c85e64df427df77f1; no voters:
I20260619 08:51:08.307883 18471 raft_consensus.cc:2781] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:08.307919 18471 raft_consensus.cc:482] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:08.307942 18471 raft_consensus.cc:3037] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:08.308640 18471 raft_consensus.cc:504] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.308825 18471 leader_election.cc:288] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [CANDIDATE]: Term 1 election: Requested vote from peers abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061), e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511)
I20260619 08:51:08.309026 18228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9fa59f48134fea9484373faf2e8ee4" candidate_uuid: "831b3bcb69b3453eb9d1ed8566176761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ad6e382e7e4f9c85e64df427df77f1"
I20260619 08:51:08.308980 18088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9fa59f48134fea9484373faf2e8ee4" candidate_uuid: "831b3bcb69b3453eb9d1ed8566176761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abd93b64f7804f70a89e70fe2c90be39"
I20260619 08:51:08.309092 18228 raft_consensus.cc:3037] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:08.309093 18088 raft_consensus.cc:3037] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:08.309855 18228 raft_consensus.cc:2454] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 831b3bcb69b3453eb9d1ed8566176761 in term 1.
I20260619 08:51:08.309856 18088 raft_consensus.cc:2454] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 831b3bcb69b3453eb9d1ed8566176761 in term 1.
I20260619 08:51:08.310057 18301 leader_election.cc:302] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 831b3bcb69b3453eb9d1ed8566176761, abd93b64f7804f70a89e70fe2c90be39; no voters:
I20260619 08:51:08.310173 18471 raft_consensus.cc:2781] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:08.310240 18471 raft_consensus.cc:686] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 1 LEADER]: Becoming Leader. State: Replica: 831b3bcb69b3453eb9d1ed8566176761, State: Running, Role: LEADER
I20260619 08:51:08.310289 18471 consensus_queue.cc:237] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.310951 17931 catalog_manager.cc:5495] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 reported cstate change: term changed from 0 to 1, leader changed from <none> to 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3). New cstate: current_term: 1 leader_uuid: "831b3bcb69b3453eb9d1ed8566176761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } health_report { overall_health: HEALTHY } } }
I20260619 08:51:08.327616 18543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:08.325304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52106 (local address 127.15.208.3:40081)
0619 08:51:08.325509 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:08.325511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:08.325519 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:08.325539 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:08.325540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:08.325551 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:08.325618 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:08.325738 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:08.326219 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:08.326976 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:08.327169 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:08.327230 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:08.327342 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:08.327402 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:08.327477 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:08.327534 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 08:51:08.327551 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260619 08:51:08.332599 18546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:08.329939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:40081 (local address 127.15.208.3:34945)
0619 08:51:08.330177 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:08.330189 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:08.330270 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:08.330406 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:08.330406 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:08.330408 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:08.330502 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:08.330504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:08.331136 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:08.331137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:08.331706 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:08.331708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:08.332160 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:08.332161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:08.332226 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:08.332230 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:08.332399 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:08.332400 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:51:08.332464 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 08:51:08.332528 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260619 08:51:08.332599 18543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:08.330230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:34945 (local address 127.15.208.3:40081)
0619 08:51:08.330283 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:08.330285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:08.330296 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:51:08.330334 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:08.330334 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:08.330346 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:08.330403 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:08.330579 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:08.331040 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:08.331815 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:08.332080 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:08.332163 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:08.332316 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:08.332347 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:08.332400 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:08.332468 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 08:51:08.332510 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":34}
I20260619 08:51:08.334662 18088 raft_consensus.cc:1261] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Refusing update from remote peer 831b3bcb69b3453eb9d1ed8566176761: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:08.334841 18228 raft_consensus.cc:1261] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Refusing update from remote peer 831b3bcb69b3453eb9d1ed8566176761: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:08.334919 18509 consensus_queue.cc:1035] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:08.335047 18471 consensus_queue.cc:1035] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:08.345986 17930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:08.346119 17930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:08.346825 17930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285: Sending DeleteTablet for 3 replicas of tablet dc9fa59f48134fea9484373faf2e8ee4
I20260619 08:51:08.347124 18068 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9fa59f48134fea9484373faf2e8ee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:08 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53592
I20260619 08:51:08.347237 18208 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9fa59f48134fea9484373faf2e8ee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:08 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:50066
I20260619 08:51:08.347289 18524 tablet_replica.cc:330] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39: stopping tablet replica
I20260619 08:51:08.347348 18524 raft_consensus.cc:2229] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:08.347393 18524 raft_consensus.cc:2258] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:08.347476 18523 tablet_replica.cc:330] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1: stopping tablet replica
I20260619 08:51:08.347530 18523 raft_consensus.cc:2229] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:08.347568 18523 raft_consensus.cc:2258] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:08.348048 18523 ts_tablet_manager.cc:1905] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:08.349673 18523 ts_tablet_manager.cc:1918] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:08.349743 18523 log.cc:1196] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/dc9fa59f48134fea9484373faf2e8ee4
I20260619 08:51:08.349731 18524 ts_tablet_manager.cc:1905] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:08.349993 18523 ts_tablet_manager.cc:1939] T dc9fa59f48134fea9484373faf2e8ee4 P e6ad6e382e7e4f9c85e64df427df77f1: Deleting consensus metadata
I20260619 08:51:08.350356 17915 catalog_manager.cc:4841] TS e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511): tablet dc9fa59f48134fea9484373faf2e8ee4 (table test-table [id=19db1d28632648879b0050a33f3871d2]) successfully deleted
I20260619 08:51:08.350865 18332 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9fa59f48134fea9484373faf2e8ee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:08 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:52090
I20260619 08:51:08.350986 18525 tablet_replica.cc:330] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761: stopping tablet replica
I20260619 08:51:08.351051 18525 raft_consensus.cc:2229] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:08.351161 18525 raft_consensus.cc:2258] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:08.351094 17930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:08.351307 17930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:08.351464 18525 ts_tablet_manager.cc:1905] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:08.352154 18524 ts_tablet_manager.cc:1918] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:08.352221 18524 log.cc:1196] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/dc9fa59f48134fea9484373faf2e8ee4
I20260619 08:51:08.352475 18524 ts_tablet_manager.cc:1939] T dc9fa59f48134fea9484373faf2e8ee4 P abd93b64f7804f70a89e70fe2c90be39: Deleting consensus metadata
I20260619 08:51:08.353134 18525 ts_tablet_manager.cc:1918] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:08.353200 18525 log.cc:1196] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/dc9fa59f48134fea9484373faf2e8ee4
I20260619 08:51:08.353428 18525 ts_tablet_manager.cc:1939] T dc9fa59f48134fea9484373faf2e8ee4 P 831b3bcb69b3453eb9d1ed8566176761: Deleting consensus metadata
I20260619 08:51:08.353885 17917 catalog_manager.cc:4841] TS 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081): tablet dc9fa59f48134fea9484373faf2e8ee4 (table test-table [id=19db1d28632648879b0050a33f3871d2]) successfully deleted
I20260619 08:51:08.353914 18208 tablet_service.cc:1467] Processing CreateTablet for tablet 7b9d7b58ff5f4f3f942d9e850f2389aa (DEFAULT_TABLE table=test-table [id=90073ffea70149c1a337d39bab9ad14d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:08.354031 18208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b9d7b58ff5f4f3f942d9e850f2389aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:08.354092 18332 tablet_service.cc:1467] Processing CreateTablet for tablet 7b9d7b58ff5f4f3f942d9e850f2389aa (DEFAULT_TABLE table=test-table [id=90073ffea70149c1a337d39bab9ad14d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:08.354092 18068 tablet_service.cc:1467] Processing CreateTablet for tablet 7b9d7b58ff5f4f3f942d9e850f2389aa (DEFAULT_TABLE table=test-table [id=90073ffea70149c1a337d39bab9ad14d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:08.354202 18332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b9d7b58ff5f4f3f942d9e850f2389aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:08.354202 18068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b9d7b58ff5f4f3f942d9e850f2389aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:08.355291 18538 tablet_bootstrap.cc:492] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1: Bootstrap starting.
I20260619 08:51:08.355448 18534 tablet_bootstrap.cc:492] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39: Bootstrap starting.
I20260619 08:51:08.355639 18538 tablet_bootstrap.cc:654] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:08.355908 18534 tablet_bootstrap.cc:654] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:08.356189 18535 tablet_bootstrap.cc:492] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761: Bootstrap starting.
I20260619 08:51:08.356359 18538 tablet_bootstrap.cc:492] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1: No bootstrap required, opened a new log
I20260619 08:51:08.356419 18538 ts_tablet_manager.cc:1397] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:08.356530 18535 tablet_bootstrap.cc:654] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:08.356526 18538 raft_consensus.cc:348] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.356627 18538 raft_consensus.cc:374] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:08.356634 18534 tablet_bootstrap.cc:492] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39: No bootstrap required, opened a new log
I20260619 08:51:08.356657 18538 raft_consensus.cc:729] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ad6e382e7e4f9c85e64df427df77f1, State: Initialized, Role: FOLLOWER
I20260619 08:51:08.356683 18534 ts_tablet_manager.cc:1397] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:08.356709 18538 consensus_queue.cc:260] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.356832 18538 ts_tablet_manager.cc:1428] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:08.356850 18534 raft_consensus.cc:348] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.356948 18534 raft_consensus.cc:374] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:08.356972 18534 raft_consensus.cc:729] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd93b64f7804f70a89e70fe2c90be39, State: Initialized, Role: FOLLOWER
I20260619 08:51:08.357028 18534 consensus_queue.cc:260] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.357164 18534 ts_tablet_manager.cc:1428] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:08.357721 17917 catalog_manager.cc:4841] TS abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061): tablet dc9fa59f48134fea9484373faf2e8ee4 (table test-table [id=19db1d28632648879b0050a33f3871d2]) successfully deleted
I20260619 08:51:08.363929 18535 tablet_bootstrap.cc:492] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761: No bootstrap required, opened a new log
I20260619 08:51:08.364012 18535 ts_tablet_manager.cc:1397] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260619 08:51:08.364135 18535 raft_consensus.cc:348] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.364233 18535 raft_consensus.cc:374] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:08.364253 18535 raft_consensus.cc:729] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831b3bcb69b3453eb9d1ed8566176761, State: Initialized, Role: FOLLOWER
I20260619 08:51:08.364318 18535 consensus_queue.cc:260] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.364413 18535 ts_tablet_manager.cc:1428] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:08.457335 18518 raft_consensus.cc:482] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:08.457449 18518 raft_consensus.cc:504] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.457615 18518 leader_election.cc:288] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061), 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081)
I20260619 08:51:08.457841 18088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b9d7b58ff5f4f3f942d9e850f2389aa" candidate_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abd93b64f7804f70a89e70fe2c90be39" is_pre_election: true
I20260619 08:51:08.457933 18368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b9d7b58ff5f4f3f942d9e850f2389aa" candidate_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831b3bcb69b3453eb9d1ed8566176761" is_pre_election: true
I20260619 08:51:08.457998 18088 raft_consensus.cc:2454] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6ad6e382e7e4f9c85e64df427df77f1 in term 0.
I20260619 08:51:08.458014 18368 raft_consensus.cc:2454] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6ad6e382e7e4f9c85e64df427df77f1 in term 0.
I20260619 08:51:08.458155 18161 leader_election.cc:302] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 831b3bcb69b3453eb9d1ed8566176761, e6ad6e382e7e4f9c85e64df427df77f1; no voters:
I20260619 08:51:08.458256 18518 raft_consensus.cc:2781] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:08.458318 18518 raft_consensus.cc:482] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:08.458345 18518 raft_consensus.cc:3037] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:08.459038 18518 raft_consensus.cc:504] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.459182 18518 leader_election.cc:288] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [CANDIDATE]: Term 1 election: Requested vote from peers abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061), 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081)
I20260619 08:51:08.459373 18088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b9d7b58ff5f4f3f942d9e850f2389aa" candidate_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abd93b64f7804f70a89e70fe2c90be39"
I20260619 08:51:08.459408 18368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b9d7b58ff5f4f3f942d9e850f2389aa" candidate_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "831b3bcb69b3453eb9d1ed8566176761"
I20260619 08:51:08.459472 18368 raft_consensus.cc:3037] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:08.459478 18088 raft_consensus.cc:3037] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:08.460088 18088 raft_consensus.cc:2454] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ad6e382e7e4f9c85e64df427df77f1 in term 1.
I20260619 08:51:08.460217 18368 raft_consensus.cc:2454] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ad6e382e7e4f9c85e64df427df77f1 in term 1.
I20260619 08:51:08.460320 18161 leader_election.cc:302] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abd93b64f7804f70a89e70fe2c90be39, e6ad6e382e7e4f9c85e64df427df77f1; no voters:
I20260619 08:51:08.460539 18518 raft_consensus.cc:2781] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:08.460621 18518 raft_consensus.cc:686] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 LEADER]: Becoming Leader. State: Replica: e6ad6e382e7e4f9c85e64df427df77f1, State: Running, Role: LEADER
I20260619 08:51:08.460671 18518 consensus_queue.cc:237] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } }
I20260619 08:51:08.461356 17930 catalog_manager.cc:5495] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2). New cstate: current_term: 1 leader_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6ad6e382e7e4f9c85e64df427df77f1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 36511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:08.479995 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:08.477617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.15.208.2:36511)
0619 08:51:08.477802 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:08.477804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:08.477811 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:51:08.477837 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:08.477837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:08.477847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:08.477916 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:08.478052 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:08.478540 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:08.479276 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:08.479489 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:08.479564 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:08.479716 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:08.479750 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:08.479816 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:08.479876 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 08:51:08.479895 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260619 08:51:08.482241 18368 raft_consensus.cc:1261] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Refusing update from remote peer e6ad6e382e7e4f9c85e64df427df77f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:08.482241 18088 raft_consensus.cc:1261] T 7b9d7b58ff5f4f3f942d9e850f2389aa P abd93b64f7804f70a89e70fe2c90be39 [term 1 FOLLOWER]: Refusing update from remote peer e6ad6e382e7e4f9c85e64df427df77f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:08.482511 18518 consensus_queue.cc:1035] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "831b3bcb69b3453eb9d1ed8566176761" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:08.482645 18500 consensus_queue.cc:1035] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abd93b64f7804f70a89e70fe2c90be39" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 35061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 08:51:08.487206 18505 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 08:51:10.004706 17929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:10.004832 17929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:10.005559 17929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6e0bade3ff14daeb99bb991e6eeb285: Sending DeleteTablet for 3 replicas of tablet 7b9d7b58ff5f4f3f942d9e850f2389aa
I20260619 08:51:10.005878 18208 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b9d7b58ff5f4f3f942d9e850f2389aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:10 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:50066
I20260619 08:51:10.005878 18332 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b9d7b58ff5f4f3f942d9e850f2389aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:10 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:52090
I20260619 08:51:10.006071 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 18003
I20260619 08:51:10.006078 18068 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b9d7b58ff5f4f3f942d9e850f2389aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:10 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53592
I20260619 08:51:10.007383 18583 tablet_replica.cc:330] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1: stopping tablet replica
I20260619 08:51:10.007465 18583 raft_consensus.cc:2229] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:10.007542 18583 raft_consensus.cc:2258] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:10.007767 18583 ts_tablet_manager.cc:1905] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:10.008841 18582 tablet_replica.cc:330] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761: stopping tablet replica
I20260619 08:51:10.008917 18582 raft_consensus.cc:2229] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:10.008962 18582 raft_consensus.cc:2258] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:10.009220 18582 ts_tablet_manager.cc:1905] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:10.009402 18583 ts_tablet_manager.cc:1918] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:51:10.009464 18583 log.cc:1196] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/7b9d7b58ff5f4f3f942d9e850f2389aa
I20260619 08:51:10.009778 18583 ts_tablet_manager.cc:1939] T 7b9d7b58ff5f4f3f942d9e850f2389aa P e6ad6e382e7e4f9c85e64df427df77f1: Deleting consensus metadata
I20260619 08:51:10.010114 17915 catalog_manager.cc:4841] TS e6ad6e382e7e4f9c85e64df427df77f1 (127.15.208.2:36511): tablet 7b9d7b58ff5f4f3f942d9e850f2389aa (table test-table [id=90073ffea70149c1a337d39bab9ad14d]) successfully deleted
I20260619 08:51:10.011118 18582 ts_tablet_manager.cc:1918] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:51:10.011185 18582 log.cc:1196] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/7b9d7b58ff5f4f3f942d9e850f2389aa
I20260619 08:51:10.011394 18582 ts_tablet_manager.cc:1939] T 7b9d7b58ff5f4f3f942d9e850f2389aa P 831b3bcb69b3453eb9d1ed8566176761: Deleting consensus metadata
I20260619 08:51:10.011741 17917 catalog_manager.cc:4841] TS 831b3bcb69b3453eb9d1ed8566176761 (127.15.208.3:40081): tablet 7b9d7b58ff5f4f3f942d9e850f2389aa (table test-table [id=90073ffea70149c1a337d39bab9ad14d]) successfully deleted
W20260619 08:51:10.012903 17917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:35061 (error 108)
I20260619 08:51:10.013093 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 18143
I20260619 08:51:10.014036 18585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:10.013824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:35061 (local address 127.0.0.1:53610)
0619 08:51:10.013942 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:10.013988 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:35061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
W20260619 08:51:10.014140 17917 catalog_manager.cc:4584] TS abd93b64f7804f70a89e70fe2c90be39 (127.15.208.1:35061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b9d7b58ff5f4f3f942d9e850f2389aa: Network error: Client connection negotiation failed: client connection to 127.15.208.1:35061: connect: Connection refused (error 111)
I20260619 08:51:10.019143 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 18283
I20260619 08:51:10.024580 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 17899
2026-06-19T08:51:10Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6046 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:10 dist-test-slave-p0h9 krb5kdc[18588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:10 dist-test-slave-p0h9 krb5kdc[18588](info): set up 2 sockets
Jun 19 08:51:10 dist-test-slave-p0h9 krb5kdc[18588](info): commencing operation
krb5kdc: starting...
W20260619 08:51:11.085050 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:11.201560 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:37741
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:37741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:11.280998 18601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:11.281201 18601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:11.281224 18601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:11.282357 18601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:11.282400 18601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:11.282415 18601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:11.282428 18601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:11.282440 18601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:11.283878 18601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:37741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:37741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:11.284067 18601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:11.284240 18601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:11.289153 18606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:11.289131 18609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:11.289134 18607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:11.289162 18601 server_base.cc:1034] running on GCE node
I20260619 08:51:11.289610 18601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:11.289945 18601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:11.291110 18601 hybrid_clock.cc:648] HybridClock initialized: now 1781859071291101 us; error 29 us; skew 500 ppm
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859071, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:11.294013 18601 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:11.294483 18601 webserver.cc:460] Webserver started at http://127.15.208.62:41349/ using document root <none> and password file <none>
I20260619 08:51:11.294735 18601 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:11.294811 18601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:11.294966 18601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:11.296056 18601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "b714a87367764f9ebf376e1b0d8dec50"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "667243494a83564a0e5b17fca9b5f7be"
server_key_iv: "57d7226cbd55d23bee06a12e16ac11d0"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.296415 18601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "b714a87367764f9ebf376e1b0d8dec50"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "667243494a83564a0e5b17fca9b5f7be"
server_key_iv: "57d7226cbd55d23bee06a12e16ac11d0"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.297856 18601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:11.298684 18616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:11.298884 18601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:51:11.298990 18601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "b714a87367764f9ebf376e1b0d8dec50"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "667243494a83564a0e5b17fca9b5f7be"
server_key_iv: "57d7226cbd55d23bee06a12e16ac11d0"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.299075 18601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:11.308568 18601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:11.310549 18601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:11.310719 18601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:11.314947 18601 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:37741
I20260619 08:51:11.315009 18668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:37741 every 8 connection(s)
I20260619 08:51:11.315364 18601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:11.316025 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 18601
I20260619 08:51:11.316109 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:11.316299 16192 external_mini_cluster.cc:1428] Setting key 4c58696360a97c6024713dd6839fdd94
I20260619 08:51:11.316305 18669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:11.319653 18669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50: Bootstrap starting.
I20260619 08:51:11.320411 18669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:11.320703 18669 log.cc:824] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:11.321470 18669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50: No bootstrap required, opened a new log
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[18588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:11.323338 18669 raft_consensus.cc:348] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b714a87367764f9ebf376e1b0d8dec50" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37741 } }
I20260619 08:51:11.323496 18669 raft_consensus.cc:374] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:11.323545 18669 raft_consensus.cc:729] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b714a87367764f9ebf376e1b0d8dec50, State: Initialized, Role: FOLLOWER
I20260619 08:51:11.323726 18669 consensus_queue.cc:260] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b714a87367764f9ebf376e1b0d8dec50" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37741 } }
I20260619 08:51:11.323820 18669 raft_consensus.cc:388] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:11.323856 18669 raft_consensus.cc:482] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:11.323897 18669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:11.324581 18669 raft_consensus.cc:504] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b714a87367764f9ebf376e1b0d8dec50" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37741 } }
I20260619 08:51:11.324715 18669 leader_election.cc:302] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b714a87367764f9ebf376e1b0d8dec50; no voters:
I20260619 08:51:11.324932 18669 leader_election.cc:288] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:11.324965 18674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:11.325096 18674 raft_consensus.cc:686] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [term 1 LEADER]: Becoming Leader. State: Replica: b714a87367764f9ebf376e1b0d8dec50, State: Running, Role: LEADER
I20260619 08:51:11.325220 18669 sys_catalog.cc:564] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:11.325238 18674 consensus_queue.cc:237] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b714a87367764f9ebf376e1b0d8dec50" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37741 } }
I20260619 08:51:11.325552 18675 sys_catalog.cc:455] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b714a87367764f9ebf376e1b0d8dec50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b714a87367764f9ebf376e1b0d8dec50" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37741 } } }
I20260619 08:51:11.325640 18675 sys_catalog.cc:458] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:11.325537 18674 sys_catalog.cc:455] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b714a87367764f9ebf376e1b0d8dec50. Latest consensus state: current_term: 1 leader_uuid: "b714a87367764f9ebf376e1b0d8dec50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b714a87367764f9ebf376e1b0d8dec50" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37741 } } }
I20260619 08:51:11.325979 18674 sys_catalog.cc:458] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:11.326288 18680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:11.326787 18680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:11.327320 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:11.316818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55402 (local address 127.15.208.62:37741)
0619 08:51:11.317017 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:11.317025 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:11.317050 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:51:11.317464 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:11.317470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:11.317513 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:11.317627 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:11.318413 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.319289 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.320333 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.320885 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.321000 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:11.323441 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:11.323446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:11.323446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:11.323459 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:11.325546 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:11.325986 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:11.325989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:11.325990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:11.326115 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:11.326349 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:11.326350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:11.326350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:11.326644 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:11.326782 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:11.326975 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 08:51:11.327128 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260619 08:51:11.328423 18680 catalog_manager.cc:1347] Generated new cluster ID: 72608acf6a124f93ac5e7085b0379cd1
I20260619 08:51:11.328471 18680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:11.344120 18680 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:11.344666 18680 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:11.354293 18680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b714a87367764f9ebf376e1b0d8dec50: Generated new TSK 0
I20260619 08:51:11.354492 18680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:51:11.389330 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37741
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:11.474342 18695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:11.474550 18695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:11.474574 18695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:11.475724 18695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:11.475762 18695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:11.475788 18695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:11.477219 18695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:11.477411 18695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:11.477576 18695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:11.482452 18700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:11.482466 18701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:11.482441 18703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:11.482488 18695 server_base.cc:1034] running on GCE node
I20260619 08:51:11.482872 18695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:11.483127 18695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:11.484305 18695 hybrid_clock.cc:648] HybridClock initialized: now 1781859071484302 us; error 38 us; skew 500 ppm
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859071, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:11.486883 18695 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:51:11.487303 18695 webserver.cc:460] Webserver started at http://127.15.208.1:41647/ using document root <none> and password file <none>
I20260619 08:51:11.487524 18695 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:11.487591 18695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:11.487740 18695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:11.488788 18695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "2b7fdb5fa8e74d0b81677d90cc601d66"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "bdff3de0f89ea8cb67396fa8af76868b"
server_key_iv: "69c1028570290c59ca237706001c94b7"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.489157 18695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "2b7fdb5fa8e74d0b81677d90cc601d66"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "bdff3de0f89ea8cb67396fa8af76868b"
server_key_iv: "69c1028570290c59ca237706001c94b7"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.490428 18695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:11.491130 18710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:11.491298 18695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:11.491393 18695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "2b7fdb5fa8e74d0b81677d90cc601d66"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "bdff3de0f89ea8cb67396fa8af76868b"
server_key_iv: "69c1028570290c59ca237706001c94b7"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.491478 18695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:11.505435 18695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:11.507275 18695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:11.507441 18695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:11.507674 18695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:11.507987 18695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:11.508046 18695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:11.508095 18695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:11.508122 18695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:11.514127 18695 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:46217
I20260619 08:51:11.514173 18823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:46217 every 8 connection(s)
I20260619 08:51:11.514540 18695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[18588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859071, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:11.523466 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:11.515290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:58577 (local address 127.15.208.62:37741)
0619 08:51:11.515379 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:11.515382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:11.515568 (+ 186us) server_negotiation.cc:407] Connection header received
0619 08:51:11.516104 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:11.516106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:11.516117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:11.516170 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:11.517228 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.517757 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.518760 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.518986 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.519073 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:11.521127 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:11.521128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:11.521128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:11.521135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:11.522320 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:11.522696 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:11.522696 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:11.522696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:11.522742 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:11.522945 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:11.522945 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:11.522945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:11.523049 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:11.523102 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:11.523338 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 08:51:11.523400 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 08:51:11.523653 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:11.515254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:37741 (local address 127.15.208.1:58577)
0619 08:51:11.515427 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:11.515477 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:11.515993 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:11.516248 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:11.516252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:11.516933 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:11.517125 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:11.517130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.517848 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.517849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:11.518653 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:11.518655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.519081 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.519082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:11.519207 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:11.519657 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:11.519665 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:11.521042 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:11.522392 (+ 1350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:11.522397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:11.522399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:11.522633 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:11.522800 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:11.522800 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:11.522801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:11.522893 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:11.523109 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:11.523113 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:11.523234 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 08:51:11.523312 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:51:11.523426 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
I20260619 08:51:11.524150 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 18695
I20260619 08:51:11.524186 18824 heartbeater.cc:347] Connected to a master server at 127.15.208.62:37741
I20260619 08:51:11.524256 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:11.524276 18824 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:11.524443 16192 external_mini_cluster.cc:1428] Setting key 97d517cad2b482e14d134582855caca1
I20260619 08:51:11.524457 18824 heartbeater.cc:511] Master 127.15.208.62:37741 requested a full tablet report, sending...
I20260619 08:51:11.525053 18632 ts_manager.cc:194] Registered new tserver with Master: 2b7fdb5fa8e74d0b81677d90cc601d66 (127.15.208.1:46217)
I20260619 08:51:11.525738 18632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:58577
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:51:11.547621 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37741
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:11.627643 18829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:11.627836 18829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:11.627858 18829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:11.629029 18829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:11.629069 18829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:11.629123 18829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:11.630532 18829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:11.630779 18829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:11.631006 18829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:11.635988 18835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:11.636018 18837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:11.635999 18834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:11.636034 18829 server_base.cc:1034] running on GCE node
I20260619 08:51:11.636376 18829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:11.636588 18829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:11.637722 18829 hybrid_clock.cc:648] HybridClock initialized: now 1781859071637711 us; error 28 us; skew 500 ppm
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859071, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:11.640342 18829 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:51:11.640844 18829 webserver.cc:460] Webserver started at http://127.15.208.2:34893/ using document root <none> and password file <none>
I20260619 08:51:11.641088 18829 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:11.641155 18829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:11.641302 18829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:11.642444 18829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "585acdd28fec4dd6937edc39d986735e"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "2d88f308a11d59cf0dfa98f081096cbb"
server_key_iv: "c55c247809922a62be62ab6602f3fa64"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.642807 18829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "585acdd28fec4dd6937edc39d986735e"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "2d88f308a11d59cf0dfa98f081096cbb"
server_key_iv: "c55c247809922a62be62ab6602f3fa64"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.644130 18829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:11.644979 18844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:11.645174 18829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:11.645272 18829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "585acdd28fec4dd6937edc39d986735e"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "2d88f308a11d59cf0dfa98f081096cbb"
server_key_iv: "c55c247809922a62be62ab6602f3fa64"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.645352 18829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:11.664268 18829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:11.666314 18829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:11.666494 18829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:11.666750 18829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:11.667078 18829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:11.667132 18829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:11.667167 18829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:11.667214 18829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:11.672931 18829 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:42527
I20260619 08:51:11.672974 18957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:42527 every 8 connection(s)
I20260619 08:51:11.673295 18829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[18588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859071, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:11.682143 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:11.673935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:60203 (local address 127.15.208.62:37741)
0619 08:51:11.673985 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:11.673988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:11.674234 (+ 246us) server_negotiation.cc:407] Connection header received
0619 08:51:11.674702 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:11.674703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:11.674713 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:11.674768 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:11.675715 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.676203 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.677235 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.677417 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.677471 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:11.679523 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:11.679524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:11.679524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:11.679531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:11.681036 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:11.681377 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:11.681378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:11.681378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:11.681427 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:11.681648 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:11.681649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:11.681649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:11.681748 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:11.681798 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:11.682017 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 08:51:11.682077 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:11.682308 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:11.673930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:37741 (local address 127.15.208.2:60203)
0619 08:51:11.674111 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:11.674147 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:11.674613 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:11.674842 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:11.674845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:11.675400 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:11.675623 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:11.675628 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.676281 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.676281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:11.677164 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:11.677166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.677486 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.677487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:11.677596 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:11.678058 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:11.678067 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:11.679433 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:11.681102 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:11.681107 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:11.681109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:11.681315 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:11.681483 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:11.681483 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:11.681484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:11.681591 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:11.681802 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:11.681806 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:11.681910 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 08:51:11.681990 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:51:11.682111 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260619 08:51:11.682718 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 18829
I20260619 08:51:11.682825 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:11.682907 18958 heartbeater.cc:347] Connected to a master server at 127.15.208.62:37741
I20260619 08:51:11.683005 18958 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:11.683039 16192 external_mini_cluster.cc:1428] Setting key 07a2d9228b3773e527d0b2daab234691
I20260619 08:51:11.683216 18958 heartbeater.cc:511] Master 127.15.208.62:37741 requested a full tablet report, sending...
I20260619 08:51:11.683627 18633 ts_manager.cc:194] Registered new tserver with Master: 585acdd28fec4dd6937edc39d986735e (127.15.208.2:42527)
I20260619 08:51:11.684049 18633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:60203
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:51:11.708442 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37741
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:11.790659 18963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:11.790854 18963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:11.790877 18963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:11.792032 18963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:11.792074 18963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:11.792104 18963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:11.793638 18963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:11.793866 18963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:11.794088 18963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:11.799120 18968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:11.799124 18971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:11.799177 18969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:11.799335 18963 server_base.cc:1034] running on GCE node
I20260619 08:51:11.799556 18963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:11.799786 18963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:11.800944 18963 hybrid_clock.cc:648] HybridClock initialized: now 1781859071800921 us; error 50 us; skew 500 ppm
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859071, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:11.803570 18963 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:51:11.804030 18963 webserver.cc:460] Webserver started at http://127.15.208.3:34231/ using document root <none> and password file <none>
I20260619 08:51:11.804251 18963 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:11.804318 18963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:11.804473 18963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:11.805594 18963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "8ca2542c96cb48eabc1d803857804aa8"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "04828f97859c74c682bff140bc1f4429"
server_key_iv: "a641ddda22fce51058749d93db73c628"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.805964 18963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "8ca2542c96cb48eabc1d803857804aa8"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "04828f97859c74c682bff140bc1f4429"
server_key_iv: "a641ddda22fce51058749d93db73c628"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.807315 18963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:51:11.808049 18978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:11.808234 18963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:11.808328 18963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "8ca2542c96cb48eabc1d803857804aa8"
format_stamp: "Formatted at 2026-06-19 08:51:11 on dist-test-slave-p0h9"
server_key: "04828f97859c74c682bff140bc1f4429"
server_key_iv: "a641ddda22fce51058749d93db73c628"
server_key_version: "encryptionkey@0"
I20260619 08:51:11.808410 18963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:11.821571 18963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:11.823932 18963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:11.824107 18963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:11.824333 18963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:11.824702 18963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:11.824775 18963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:11.824819 18963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:11.824848 18963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:11.831316 18963 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:38519
I20260619 08:51:11.831358 19091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:38519 every 8 connection(s)
I20260619 08:51:11.831712 18963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:11.833375 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 18963
I20260619 08:51:11.833483 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:11.833700 16192 external_mini_cluster.cc:1428] Setting key 2ea8a5bdafb65eeca895db6a96356e03
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[18588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859071, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:11.841171 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:11.832441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:47901 (local address 127.15.208.62:37741)
0619 08:51:11.832491 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:11.832493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:11.832694 (+ 201us) server_negotiation.cc:407] Connection header received
0619 08:51:11.833215 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:11.833216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:11.833226 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:11.833277 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:11.834592 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.835168 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.836135 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.836325 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.836376 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:11.838548 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:11.838549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:11.838549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:11.838556 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:11.839811 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:11.840182 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:11.840183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:11.840183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:11.840233 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:11.840485 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:11.840486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:11.840486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:11.840638 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:11.840701 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:11.840989 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 08:51:11.841058 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:51:11.841301 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:11.832428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:37741 (local address 127.15.208.3:47901)
0619 08:51:11.832567 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:11.832602 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:11.833115 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:11.833354 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:11.833359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:11.834240 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:11.834504 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:11.834511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.835250 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.835251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:11.836061 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:11.836063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:11.836394 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:11.836395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:11.836488 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:11.837078 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:11.837086 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:11.838418 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:11.839886 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:11.839891 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:11.839893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:11.840116 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:11.840290 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:11.840290 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:11.840291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:11.840404 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:11.840709 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:11.840714 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:11.840872 (+ 158us) client_negotiation.cc:769] Sending connection context
0619 08:51:11.840962 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:51:11.841101 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260619 08:51:11.841812 19092 heartbeater.cc:347] Connected to a master server at 127.15.208.62:37741
I20260619 08:51:11.841902 19092 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:11.842046 19092 heartbeater.cc:511] Master 127.15.208.62:37741 requested a full tablet report, sending...
I20260619 08:51:11.842382 18633 ts_manager.cc:194] Registered new tserver with Master: 8ca2542c96cb48eabc1d803857804aa8 (127.15.208.3:38519)
I20260619 08:51:11.842716 18633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:47901
I20260619 08:51:11.845439 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 08:51:11.856504 18672 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:11.855505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40260 (local address 127.15.208.62:37741)
0619 08:51:11.855614 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:11.855617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:11.855715 (+ 98us) server_negotiation.cc:407] Connection header received
0619 08:51:11.855810 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:11.855811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:11.855822 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:11.855894 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:11.856427 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40260: BlockingRecv error: recv got EOF from 127.0.0.1:40260 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:51:11.857386 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 18695
I20260619 08:51:11.861914 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 18829
I20260619 08:51:11.866215 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 18963
I20260619 08:51:11.870287 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 18601
2026-06-19T08:51:11Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1844 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[19104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[19104](info): set up 2 sockets
Jun 19 08:51:11 dist-test-slave-p0h9 krb5kdc[19104](info): commencing operation
krb5kdc: starting...
W20260619 08:51:12.904989 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:12 dist-test-slave-p0h9 krb5kdc[19104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:13.017385 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:33927
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:33927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:13.097080 19117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:13.097290 19117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:13.097312 19117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:13.098395 19117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:13.098428 19117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:13.098443 19117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:13.098454 19117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:13.098466 19117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:13.099888 19117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:33927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:33927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:13.100072 19117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:13.100248 19117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:13.105051 19123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:13.105016 19125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:13.105109 19122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:13.105209 19117 server_base.cc:1034] running on GCE node
I20260619 08:51:13.105451 19117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:13.105773 19117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:13.106926 19117 hybrid_clock.cc:648] HybridClock initialized: now 1781859073106907 us; error 40 us; skew 500 ppm
Jun 19 08:51:13 dist-test-slave-p0h9 krb5kdc[19104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859073, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:13.113979 19117 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:13.114482 19117 webserver.cc:460] Webserver started at http://127.15.208.62:46657/ using document root <none> and password file <none>
I20260619 08:51:13.114719 19117 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:13.114789 19117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:13.114944 19117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:13.116029 19117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "1d27190e970b4a9e8174446338e4875d"
format_stamp: "Formatted at 2026-06-19 08:51:13 on dist-test-slave-p0h9"
server_key: "ed4f96e6ad4f3192b837254ab05d84ae"
server_key_iv: "a3d9f2cc3e354622a72aa80eb0d131b9"
server_key_version: "encryptionkey@0"
I20260619 08:51:13.116398 19117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "1d27190e970b4a9e8174446338e4875d"
format_stamp: "Formatted at 2026-06-19 08:51:13 on dist-test-slave-p0h9"
server_key: "ed4f96e6ad4f3192b837254ab05d84ae"
server_key_iv: "a3d9f2cc3e354622a72aa80eb0d131b9"
server_key_version: "encryptionkey@0"
I20260619 08:51:13.117841 19117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:13.118603 19132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:13.118775 19117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:13.118878 19117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "1d27190e970b4a9e8174446338e4875d"
format_stamp: "Formatted at 2026-06-19 08:51:13 on dist-test-slave-p0h9"
server_key: "ed4f96e6ad4f3192b837254ab05d84ae"
server_key_iv: "a3d9f2cc3e354622a72aa80eb0d131b9"
server_key_version: "encryptionkey@0"
I20260619 08:51:13.118960 19117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:13.139663 19117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:13.141696 19117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:13.141865 19117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:13.145982 19117 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:33927
I20260619 08:51:13.146039 19184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:33927 every 8 connection(s)
I20260619 08:51:13.146539 19117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:13.147212 19185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:13.149575 19185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d: Bootstrap starting.
I20260619 08:51:13.150187 19185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:13.150482 19185 log.cc:824] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:13.151168 19185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d: No bootstrap required, opened a new log
I20260619 08:51:13.152280 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 19117
I20260619 08:51:13.152392 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:13.152576 16192 external_mini_cluster.cc:1428] Setting key c765bccc87651bb8921d0f609a77ae84
I20260619 08:51:13.153043 19185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d27190e970b4a9e8174446338e4875d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33927 } }
I20260619 08:51:13.153244 19185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:13.153294 19185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d27190e970b4a9e8174446338e4875d, State: Initialized, Role: FOLLOWER
I20260619 08:51:13.153431 19185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d27190e970b4a9e8174446338e4875d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33927 } }
I20260619 08:51:13.153501 19185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:13.153522 19185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:13.153578 19185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:13.154186 19185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d27190e970b4a9e8174446338e4875d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33927 } }
I20260619 08:51:13.154300 19185 leader_election.cc:302] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d27190e970b4a9e8174446338e4875d; no voters:
I20260619 08:51:13.154476 19185 leader_election.cc:288] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:13.154546 19190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:13.154707 19190 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [term 1 LEADER]: Becoming Leader. State: Replica: 1d27190e970b4a9e8174446338e4875d, State: Running, Role: LEADER
I20260619 08:51:13.154749 19185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:13.154793 19190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d27190e970b4a9e8174446338e4875d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33927 } }
I20260619 08:51:13.155229 19191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d27190e970b4a9e8174446338e4875d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d27190e970b4a9e8174446338e4875d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33927 } } }
I20260619 08:51:13.155215 19192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d27190e970b4a9e8174446338e4875d. Latest consensus state: current_term: 1 leader_uuid: "1d27190e970b4a9e8174446338e4875d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d27190e970b4a9e8174446338e4875d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33927 } } }
I20260619 08:51:13.155476 19191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:13.155483 19192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d [sys.catalog]: This master's current role is: LEADER
W20260619 08:51:13.156903 19206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:51:13.156960 19206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:51:13.157011 19197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:13.157155 19197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:13.158689 19197 catalog_manager.cc:1347] Generated new cluster ID: ddaae91972ee4f1492e4c42eb5700afc
I20260619 08:51:13.158735 19197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:51:13 dist-test-slave-p0h9 krb5kdc[19104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:13.163961 19197 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:13.164628 19197 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:13.165123 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:13.153099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.15.208.62:33927)
0619 08:51:13.153733 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:13.153744 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:13.153774 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:51:13.153864 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:13.153869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:13.153911 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:13.154018 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:13.154882 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:13.156490 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:13.157551 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:13.158039 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:13.158183 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:13.161241 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:13.161245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:13.161246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:13.161257 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:13.163350 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:13.163780 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:13.163784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:13.163784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:13.163899 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:13.164123 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:13.164124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:13.164125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:13.164400 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:13.164554 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:13.164669 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 08:51:13.164896 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":554,"threads_started":1}
I20260619 08:51:13.166035 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:51:13.167402 19197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d27190e970b4a9e8174446338e4875d: Generated new TSK 0
I20260619 08:51:13.167528 19197 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:51:13.180332 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:13.176934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.15.208.62:33927)
0619 08:51:13.177021 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:13.177023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:13.177030 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:51:13.177050 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:13.177051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:13.177060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:13.177128 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:13.177497 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:13.177999 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:13.178766 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:13.178977 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:13.179068 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:13.179796 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:13.179797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:13.179798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:13.180034 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:13.180052 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:13.180058 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:13.180142 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:13.180213 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 08:51:13.180239 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 08:51:13.182622 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 19117
2026-06-19T08:51:13Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1310 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:13 dist-test-slave-p0h9 krb5kdc[19218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:13 dist-test-slave-p0h9 krb5kdc[19218](info): set up 2 sockets
Jun 19 08:51:13 dist-test-slave-p0h9 krb5kdc[19218](info): commencing operation
krb5kdc: starting...
W20260619 08:51:14.293284 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:14 dist-test-slave-p0h9 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:14.411623 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:42871
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:42871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:14.497138 19231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:14.497334 19231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:14.497355 19231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:14.498494 19231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:14.498553 19231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:14.498567 19231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:14.498580 19231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:14.498592 19231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:14.500042 19231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:42871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:42871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:14.500290 19231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:14.500555 19231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:14.505506 19237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:14.505453 19239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:14.505450 19236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:14.505771 19231 server_base.cc:1034] running on GCE node
I20260619 08:51:14.506090 19231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:14.506417 19231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:14.507613 19231 hybrid_clock.cc:648] HybridClock initialized: now 1781859074507604 us; error 64 us; skew 500 ppm
Jun 19 08:51:14 dist-test-slave-p0h9 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859074, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:14.510340 19231 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:14.510857 19231 webserver.cc:460] Webserver started at http://127.15.208.62:42391/ using document root <none> and password file <none>
I20260619 08:51:14.511125 19231 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:14.511201 19231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:14.511363 19231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:14.512537 19231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "d0a0eb4deeec477690bbfd5350047f30"
format_stamp: "Formatted at 2026-06-19 08:51:14 on dist-test-slave-p0h9"
server_key: "edf3351ec0f4a76681bf0edc58ded105"
server_key_iv: "41616df03eaeb93e6a4d39a97541e897"
server_key_version: "encryptionkey@0"
I20260619 08:51:14.512975 19231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "d0a0eb4deeec477690bbfd5350047f30"
format_stamp: "Formatted at 2026-06-19 08:51:14 on dist-test-slave-p0h9"
server_key: "edf3351ec0f4a76681bf0edc58ded105"
server_key_iv: "41616df03eaeb93e6a4d39a97541e897"
server_key_version: "encryptionkey@0"
I20260619 08:51:14.514389 19231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:14.515213 19246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:14.515386 19231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:14.515542 19231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "d0a0eb4deeec477690bbfd5350047f30"
format_stamp: "Formatted at 2026-06-19 08:51:14 on dist-test-slave-p0h9"
server_key: "edf3351ec0f4a76681bf0edc58ded105"
server_key_iv: "41616df03eaeb93e6a4d39a97541e897"
server_key_version: "encryptionkey@0"
I20260619 08:51:14.515625 19231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:14.540807 19231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:14.542858 19231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:14.543052 19231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:14.547732 19231 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:42871
I20260619 08:51:14.547767 19298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:42871 every 8 connection(s)
I20260619 08:51:14.548154 19231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:14.549240 19299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:14.551525 19299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30: Bootstrap starting.
I20260619 08:51:14.552124 19299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:14.552412 19299 log.cc:824] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:14.553272 19299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30: No bootstrap required, opened a new log
I20260619 08:51:14.554832 19299 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0eb4deeec477690bbfd5350047f30" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42871 } }
I20260619 08:51:14.554971 19299 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:14.554993 19299 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0a0eb4deeec477690bbfd5350047f30, State: Initialized, Role: FOLLOWER
I20260619 08:51:14.555225 19299 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0eb4deeec477690bbfd5350047f30" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42871 } }
I20260619 08:51:14.555290 19299 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:14.555311 19299 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:14.555371 19299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:14.556159 19299 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0eb4deeec477690bbfd5350047f30" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42871 } }
I20260619 08:51:14.556241 19299 leader_election.cc:302] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0a0eb4deeec477690bbfd5350047f30; no voters:
I20260619 08:51:14.556440 19299 leader_election.cc:288] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:14.556564 19302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:14.556731 19299 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:14.556748 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 19231
I20260619 08:51:14.556777 19302 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [term 1 LEADER]: Becoming Leader. State: Replica: d0a0eb4deeec477690bbfd5350047f30, State: Running, Role: LEADER
I20260619 08:51:14.556864 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:14.556885 19302 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0eb4deeec477690bbfd5350047f30" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42871 } }
I20260619 08:51:14.557082 16192 external_mini_cluster.cc:1428] Setting key c7d91f34eade8d4cab9524f672f4fb2f
I20260619 08:51:14.557281 19303 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0a0eb4deeec477690bbfd5350047f30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0eb4deeec477690bbfd5350047f30" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42871 } } }
I20260619 08:51:14.557385 19303 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:14.557621 19304 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0a0eb4deeec477690bbfd5350047f30. Latest consensus state: current_term: 1 leader_uuid: "d0a0eb4deeec477690bbfd5350047f30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0eb4deeec477690bbfd5350047f30" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42871 } } }
I20260619 08:51:14.557705 19304 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:14.557853 19311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:14.558429 19311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:14.560153 19311 catalog_manager.cc:1347] Generated new cluster ID: aa94c6d8736d4140b46f02ac6aafc11d
I20260619 08:51:14.560210 19311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:51:14 dist-test-slave-p0h9 krb5kdc[19218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:14.570046 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:14.557774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.15.208.62:42871)
0619 08:51:14.558251 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:14.558262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:14.558293 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:51:14.558422 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:14.558430 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:14.558496 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:14.558897 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:14.559614 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:14.561191 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:14.562454 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:14.563091 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:14.563210 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:14.566146 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:14.566152 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:14.566153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:14.566168 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:14.568262 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:14.568671 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:14.568675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:14.568675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:14.568842 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:14.569046 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:14.569047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:14.569047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:14.569391 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:14.569578 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:14.569674 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 08:51:14.569824 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":97,"threads_started":1}
I20260619 08:51:14.575649 19311 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:14.576177 19311 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:14.581311 19311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0a0eb4deeec477690bbfd5350047f30: Generated new TSK 0
I20260619 08:51:14.581489 19311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:51:14.611668 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42871
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:14.694444 19325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:14.694656 19325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:14.694681 19325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:14.695712 19325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:14.695751 19325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:14.695778 19325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:14.697326 19325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:14.697532 19325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:14.697714 19325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:14.702720 19333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:14.702764 19331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:14.702860 19325 server_base.cc:1034] running on GCE node
W20260619 08:51:14.702781 19330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:14.703094 19325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:14.703383 19325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:14.704559 19325 hybrid_clock.cc:648] HybridClock initialized: now 1781859074704537 us; error 34 us; skew 500 ppm
Jun 19 08:51:14 dist-test-slave-p0h9 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859074, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:14.707265 19325 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:51:14.707712 19325 webserver.cc:460] Webserver started at http://127.15.208.1:35729/ using document root <none> and password file <none>
I20260619 08:51:14.707957 19325 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:14.708050 19325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:14.708194 19325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:14.709321 19325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "d5bf92769709491cbd76460a161ca1cc"
format_stamp: "Formatted at 2026-06-19 08:51:14 on dist-test-slave-p0h9"
server_key: "f0494194bda9354daf1e87ae9b73d471"
server_key_iv: "91adbc088aebc77dac180229091d413b"
server_key_version: "encryptionkey@0"
I20260619 08:51:14.709690 19325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "d5bf92769709491cbd76460a161ca1cc"
format_stamp: "Formatted at 2026-06-19 08:51:14 on dist-test-slave-p0h9"
server_key: "f0494194bda9354daf1e87ae9b73d471"
server_key_iv: "91adbc088aebc77dac180229091d413b"
server_key_version: "encryptionkey@0"
I20260619 08:51:14.710989 19325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:14.711719 19340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:14.711884 19325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:14.711972 19325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "d5bf92769709491cbd76460a161ca1cc"
format_stamp: "Formatted at 2026-06-19 08:51:14 on dist-test-slave-p0h9"
server_key: "f0494194bda9354daf1e87ae9b73d471"
server_key_iv: "91adbc088aebc77dac180229091d413b"
server_key_version: "encryptionkey@0"
I20260619 08:51:14.712062 19325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:14.743016 19325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:14.745075 19325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:14.745239 19325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:14.745483 19325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:14.745810 19325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:14.745864 19325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:14.745900 19325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:14.745929 19325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:14.751554 19325 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:37237
I20260619 08:51:14.751605 19453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:37237 every 8 connection(s)
I20260619 08:51:14.751955 19325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:14.756933 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 19325
I20260619 08:51:14.757019 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:14.757237 16192 external_mini_cluster.cc:1428] Setting key da636bbe97831f678534ad84b159fe5b
Jun 19 08:51:14 dist-test-slave-p0h9 krb5kdc[19218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859074, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:14.762400 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:14.752677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:42871 (local address 127.15.208.1:43091)
0619 08:51:14.752878 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:14.752913 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:14.753391 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:14.753595 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:14.753598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:14.754283 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:14.754600 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:14.754608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:14.755495 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:14.755496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:14.756305 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:14.756307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:14.756686 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:14.756687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:14.756841 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:14.757455 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:14.757466 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:14.759018 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:14.760984 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:14.760990 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:14.760991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:14.761207 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:14.761455 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:14.761456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:14.761457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:14.761597 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:14.761896 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:14.761901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:14.762032 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 08:51:14.762123 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 08:51:14.762241 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":138,"threads_started":1}
I20260619 08:51:14.762404 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:14.752745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:43091 (local address 127.15.208.62:42871)
0619 08:51:14.752802 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:14.752805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:14.753000 (+ 195us) server_negotiation.cc:407] Connection header received
0619 08:51:14.753474 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:14.753475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:14.753489 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:14.753543 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:14.754696 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:14.755408 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:14.756384 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:14.756576 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:14.756675 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:14.759113 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:14.759115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:14.759116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:14.759124 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:14.760888 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:14.761292 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:14.761293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:14.761294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:14.761360 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:14.761672 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:14.761673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:14.761674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:14.761822 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:14.761897 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:14.762171 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 08:51:14.762267 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:51:14.763012 19454 heartbeater.cc:347] Connected to a master server at 127.15.208.62:42871
I20260619 08:51:14.763129 19454 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:14.763355 19454 heartbeater.cc:511] Master 127.15.208.62:42871 requested a full tablet report, sending...
I20260619 08:51:14.763897 19263 ts_manager.cc:194] Registered new tserver with Master: d5bf92769709491cbd76460a161ca1cc (127.15.208.1:37237)
I20260619 08:51:14.764525 19263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:43091
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:51:14.783432 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42871
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:14.863746 19459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:14.863981 19459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:14.864019 19459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:14.865252 19459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:14.865314 19459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:14.865355 19459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:14.866870 19459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:14.867164 19459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:14.867400 19459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:14.872275 19464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:14.872269 19467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:14.872279 19465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:14.872614 19459 server_base.cc:1034] running on GCE node
I20260619 08:51:14.872850 19459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:14.873052 19459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:14.874189 19459 hybrid_clock.cc:648] HybridClock initialized: now 1781859074874178 us; error 30 us; skew 500 ppm
Jun 19 08:51:14 dist-test-slave-p0h9 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859074, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:14.876603 19459 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:51:14.877069 19459 webserver.cc:460] Webserver started at http://127.15.208.2:44731/ using document root <none> and password file <none>
I20260619 08:51:14.877300 19459 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:14.877369 19459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:14.877524 19459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:14.878605 19459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "504702b33b164f9ea9ad20a3a5c2f08c"
format_stamp: "Formatted at 2026-06-19 08:51:14 on dist-test-slave-p0h9"
server_key: "96e1232e26902b6928de22e96a98a9b1"
server_key_iv: "c0196c60785c132bcf93ebb6913f156a"
server_key_version: "encryptionkey@0"
I20260619 08:51:14.878973 19459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "504702b33b164f9ea9ad20a3a5c2f08c"
format_stamp: "Formatted at 2026-06-19 08:51:14 on dist-test-slave-p0h9"
server_key: "96e1232e26902b6928de22e96a98a9b1"
server_key_iv: "c0196c60785c132bcf93ebb6913f156a"
server_key_version: "encryptionkey@0"
I20260619 08:51:14.880239 19459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:14.881007 19474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:14.881181 19459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:14.881278 19459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "504702b33b164f9ea9ad20a3a5c2f08c"
format_stamp: "Formatted at 2026-06-19 08:51:14 on dist-test-slave-p0h9"
server_key: "96e1232e26902b6928de22e96a98a9b1"
server_key_iv: "c0196c60785c132bcf93ebb6913f156a"
server_key_version: "encryptionkey@0"
I20260619 08:51:14.881353 19459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:14.907418 19459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:14.909842 19459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:14.910022 19459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:14.910260 19459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:14.910591 19459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:14.910651 19459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:14.910692 19459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:14.910720 19459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:14.916512 19459 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:43561
I20260619 08:51:14.916548 19587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:43561 every 8 connection(s)
I20260619 08:51:14.916975 19459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:51:14.918463 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 19459
I20260619 08:51:14.918601 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:14.918864 16192 external_mini_cluster.cc:1428] Setting key bccb09040cba014302f408c340b2839b
Jun 19 08:51:14 dist-test-slave-p0h9 krb5kdc[19218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859074, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
I20260619 08:51:14.927244 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:14.917696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:40849 (local address 127.15.208.62:42871)
0619 08:51:14.917744 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:14.917746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:14.917952 (+ 206us) server_negotiation.cc:407] Connection header received
0619 08:51:14.918443 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:14.918445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:14.918459 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:14.918530 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:14.919678 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:14.920433 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:14.921884 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:14.922081 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:14.922183 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:14.924343 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:14.924345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:14.924345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:14.924353 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:14.925813 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:14.926234 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:14.926236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:14.926236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:14.926296 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:14.926543 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:14.926544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:14.926544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:14.926663 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:14.926779 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:14.927067 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 08:51:14.927155 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:51:14.927429 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:14.917665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:42871 (local address 127.15.208.2:40849)
0619 08:51:14.917826 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:14.917871 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:14.918348 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:14.918626 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:14.918630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:14.919373 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:14.919596 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:14.919600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:14.920520 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:14.920521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:14.921791 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:14.921794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:14.922194 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:14.922195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:14.922332 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:14.922873 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:14.922881 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:14.924229 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:14.925910 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:14.925917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:14.925920 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:14.926142 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:14.926362 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:14.926363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:14.926363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:14.926473 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:14.926772 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:14.926778 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:14.926924 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 08:51:14.927038 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 08:51:14.927196 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260619 08:51:14.927938 19588 heartbeater.cc:347] Connected to a master server at 127.15.208.62:42871
I20260619 08:51:14.928037 19588 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:14.928229 19588 heartbeater.cc:511] Master 127.15.208.62:42871 requested a full tablet report, sending...
I20260619 08:51:14.928627 19263 ts_manager.cc:194] Registered new tserver with Master: 504702b33b164f9ea9ad20a3a5c2f08c (127.15.208.2:43561)
I20260619 08:51:14.929138 19263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:40849
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:51:14.943575 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42871
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:15.025775 19593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:15.025972 19593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:15.025995 19593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:15.027143 19593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:15.027185 19593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:15.027210 19593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:15.028661 19593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:15.028916 19593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:15.029148 19593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:15.033897 19598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:15.033905 19601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:15.033910 19599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:15.034145 19593 server_base.cc:1034] running on GCE node
I20260619 08:51:15.034333 19593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:15.034571 19593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:15.035734 19593 hybrid_clock.cc:648] HybridClock initialized: now 1781859075035730 us; error 36 us; skew 500 ppm
Jun 19 08:51:15 dist-test-slave-p0h9 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859075, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:15.038440 19593 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:51:15.039005 19593 webserver.cc:460] Webserver started at http://127.15.208.3:37137/ using document root <none> and password file <none>
I20260619 08:51:15.039286 19593 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:15.039366 19593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:15.039587 19593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:15.040871 19593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "ff836e11112d44a99c29ba03f57e1fec"
format_stamp: "Formatted at 2026-06-19 08:51:15 on dist-test-slave-p0h9"
server_key: "af2359d235d4714bc834d20d7c587476"
server_key_iv: "29bb5524123d09d8f4a1ecdc1727a660"
server_key_version: "encryptionkey@0"
I20260619 08:51:15.041291 19593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "ff836e11112d44a99c29ba03f57e1fec"
format_stamp: "Formatted at 2026-06-19 08:51:15 on dist-test-slave-p0h9"
server_key: "af2359d235d4714bc834d20d7c587476"
server_key_iv: "29bb5524123d09d8f4a1ecdc1727a660"
server_key_version: "encryptionkey@0"
I20260619 08:51:15.042778 19593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:15.043620 19608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:15.043800 19593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:15.043902 19593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "ff836e11112d44a99c29ba03f57e1fec"
format_stamp: "Formatted at 2026-06-19 08:51:15 on dist-test-slave-p0h9"
server_key: "af2359d235d4714bc834d20d7c587476"
server_key_iv: "29bb5524123d09d8f4a1ecdc1727a660"
server_key_version: "encryptionkey@0"
I20260619 08:51:15.044008 19593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:15.069799 19593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:15.072235 19593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:15.072407 19593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:15.072657 19593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:15.073024 19593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:15.073081 19593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:15.073128 19593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:15.073160 19593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:15.079113 19593 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:45313
I20260619 08:51:15.079155 19721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:45313 every 8 connection(s)
I20260619 08:51:15.079540 19593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:51:15 dist-test-slave-p0h9 krb5kdc[19218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859075, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:15.088485 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 19593
I20260619 08:51:15.088600 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:15.088824 16192 external_mini_cluster.cc:1428] Setting key 850973f81ffe5b61e21ef82756725e5c
I20260619 08:51:15.089138 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:15.080294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:53795 (local address 127.15.208.62:42871)
0619 08:51:15.080360 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:15.080362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:15.080577 (+ 215us) server_negotiation.cc:407] Connection header received
0619 08:51:15.081189 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:15.081191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:15.081204 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:15.081255 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:15.082284 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:15.082793 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:15.083846 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:15.084064 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:15.084141 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:15.086273 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:15.086274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:15.086275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:15.086282 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:15.087534 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:15.087947 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:15.087948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:15.087948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:15.088009 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:15.088277 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:15.088278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:15.088279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:15.088423 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:15.088501 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:15.088934 (+ 433us) server_negotiation.cc:299] Negotiation successful
0619 08:51:15.089021 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:51:15.089339 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:15.080277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:42871 (local address 127.15.208.3:53795)
0619 08:51:15.080451 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:15.080486 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:15.081050 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:15.081332 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:15.081337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:15.081977 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:15.082179 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:15.082183 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:15.082896 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:15.082897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:15.083719 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:15.083721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:15.084159 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:15.084160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:15.084284 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:15.084793 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:15.084802 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:15.086175 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:15.087634 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:15.087641 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:15.087643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:15.087857 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:15.088094 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:15.088095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:15.088096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:15.088204 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:15.088511 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:15.088516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:15.088720 (+ 204us) client_negotiation.cc:769] Sending connection context
0619 08:51:15.088887 (+ 167us) client_negotiation.cc:240] Negotiation successful
0619 08:51:15.089073 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
I20260619 08:51:15.089792 19722 heartbeater.cc:347] Connected to a master server at 127.15.208.62:42871
I20260619 08:51:15.089890 19722 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:15.090073 19722 heartbeater.cc:511] Master 127.15.208.62:42871 requested a full tablet report, sending...
I20260619 08:51:15.090497 19263 ts_manager.cc:194] Registered new tserver with Master: ff836e11112d44a99c29ba03f57e1fec (127.15.208.3:45313)
I20260619 08:51:15.090879 19263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:53795
I20260619 08:51:15.101063 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 08:51:15 dist-test-slave-p0h9 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859075, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 08:51:15 dist-test-slave-p0h9 krb5kdc[19218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859075, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:15.123816 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:15.116612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53116 (local address 127.15.208.62:42871)
0619 08:51:15.116699 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:15.116701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:15.116828 (+ 127us) server_negotiation.cc:407] Connection header received
0619 08:51:15.116925 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:15.116926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:15.116936 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:15.117006 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:15.117605 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:15.118112 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:15.118959 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:15.119161 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:15.119236 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:15.121599 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:15.121601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:15.121601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:15.121607 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:15.122793 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:15.123105 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:15.123106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:15.123106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:15.123154 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:15.123323 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:15.123323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:15.123324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:15.123460 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:15.123534 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:15.123635 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 08:51:15.123701 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260619 08:51:15.124125 19263 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53116
I20260619 08:51:15.124603 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 19325
I20260619 08:51:15.129364 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 19459
I20260619 08:51:15.133793 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 19593
I20260619 08:51:15.138254 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 19231
2026-06-19T08:51:15Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1957 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:15 dist-test-slave-p0h9 krb5kdc[19734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:15 dist-test-slave-p0h9 krb5kdc[19734](info): set up 2 sockets
Jun 19 08:51:15 dist-test-slave-p0h9 krb5kdc[19734](info): commencing operation
krb5kdc: starting...
W20260619 08:51:16.229138 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:16.345533 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:35657
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:35657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:16.431340 19747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:16.431627 19747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:16.431668 19747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:16.432879 19747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:16.432960 19747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:16.432983 19747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:16.433005 19747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:16.433033 19747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:16.434587 19747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:35657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:35657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:16.434862 19747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:16.435097 19747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:16.440043 19752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:16.440038 19753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:16.440040 19755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:16.440279 19747 server_base.cc:1034] running on GCE node
I20260619 08:51:16.440548 19747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:16.440881 19747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:16.442065 19747 hybrid_clock.cc:648] HybridClock initialized: now 1781859076442066 us; error 42 us; skew 500 ppm
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:16.444589 19747 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:16.445096 19747 webserver.cc:460] Webserver started at http://127.15.208.62:39935/ using document root <none> and password file <none>
I20260619 08:51:16.445348 19747 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:16.445417 19747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:16.445565 19747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:16.446659 19747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "de1d194cdd0a4075856cb585bf02c308"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "30d4b98d5841f9b126f311c4fe402181"
server_key_iv: "c6275789f4263bcfb54647b33410e4a9"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.447041 19747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "de1d194cdd0a4075856cb585bf02c308"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "30d4b98d5841f9b126f311c4fe402181"
server_key_iv: "c6275789f4263bcfb54647b33410e4a9"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.448413 19747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:16.449282 19762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.449453 19747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.449558 19747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "de1d194cdd0a4075856cb585bf02c308"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "30d4b98d5841f9b126f311c4fe402181"
server_key_iv: "c6275789f4263bcfb54647b33410e4a9"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.449647 19747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:16.470918 19747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:16.473088 19747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:16.473259 19747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:16.477227 19747 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:35657
I20260619 08:51:16.477269 19814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:35657 every 8 connection(s)
I20260619 08:51:16.477773 19747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:16.478430 19815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:16.480249 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 19747
I20260619 08:51:16.480358 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:16.480552 16192 external_mini_cluster.cc:1428] Setting key 1afe93a7726bd39b0cd93beed46a0bab
I20260619 08:51:16.481155 19815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308: Bootstrap starting.
I20260619 08:51:16.481895 19815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:16.482226 19815 log.cc:824] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:16.482964 19815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308: No bootstrap required, opened a new log
I20260619 08:51:16.485347 19815 raft_consensus.cc:348] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d194cdd0a4075856cb585bf02c308" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35657 } }
I20260619 08:51:16.485512 19815 raft_consensus.cc:374] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:16.485546 19815 raft_consensus.cc:729] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1d194cdd0a4075856cb585bf02c308, State: Initialized, Role: FOLLOWER
I20260619 08:51:16.485684 19815 consensus_queue.cc:260] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d194cdd0a4075856cb585bf02c308" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35657 } }
I20260619 08:51:16.485827 19815 raft_consensus.cc:388] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:16.485879 19815 raft_consensus.cc:482] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:16.485925 19815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:16.486629 19815 raft_consensus.cc:504] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d194cdd0a4075856cb585bf02c308" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35657 } }
I20260619 08:51:16.486719 19815 leader_election.cc:302] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de1d194cdd0a4075856cb585bf02c308; no voters:
I20260619 08:51:16.486919 19815 leader_election.cc:288] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:16.486948 19820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:16.487061 19820 raft_consensus.cc:686] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [term 1 LEADER]: Becoming Leader. State: Replica: de1d194cdd0a4075856cb585bf02c308, State: Running, Role: LEADER
I20260619 08:51:16.487149 19820 consensus_queue.cc:237] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d194cdd0a4075856cb585bf02c308" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35657 } }
I20260619 08:51:16.487179 19815 sys_catalog.cc:564] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:16.487459 19820 sys_catalog.cc:455] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de1d194cdd0a4075856cb585bf02c308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d194cdd0a4075856cb585bf02c308" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35657 } } }
I20260619 08:51:16.487607 19820 sys_catalog.cc:458] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:16.487798 19820 sys_catalog.cc:455] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de1d194cdd0a4075856cb585bf02c308. Latest consensus state: current_term: 1 leader_uuid: "de1d194cdd0a4075856cb585bf02c308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d194cdd0a4075856cb585bf02c308" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35657 } } }
I20260619 08:51:16.487921 19820 sys_catalog.cc:458] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308 [sys.catalog]: This master's current role is: LEADER
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:16.488229 19827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:16.488711 19827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:16.490339 19827 catalog_manager.cc:1347] Generated new cluster ID: 1a499e362b6145b28e656a96950f7650
I20260619 08:51:16.490411 19827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:16.495409 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:16.481152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36746 (local address 127.15.208.62:35657)
0619 08:51:16.481363 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:16.481372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:16.481402 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:51:16.481995 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:16.482006 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:16.482091 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:16.482248 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:16.482863 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.484264 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.485079 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.485687 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.485792 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:16.488781 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:16.488785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:16.488785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:16.488797 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:16.493676 (+ 4879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:16.494183 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:16.494186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:16.494186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:16.494293 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:16.494486 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:16.494487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:16.494487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:16.494767 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:16.494917 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:16.495060 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 08:51:16.495187 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":83,"threads_started":1}
I20260619 08:51:16.498739 19827 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:16.499256 19827 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:16.506143 19827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de1d194cdd0a4075856cb585bf02c308: Generated new TSK 0
I20260619 08:51:16.506328 19827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:51:16.537122 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:35657
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:16.618244 19841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:16.618459 19841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:16.618484 19841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:16.619653 19841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:16.619695 19841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:16.619721 19841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:16.621238 19841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:16.621516 19841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:16.621752 19841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:16.626722 19846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:16.626761 19847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:16.626739 19841 server_base.cc:1034] running on GCE node
W20260619 08:51:16.626722 19849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:16.627169 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:16.627363 19841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:16.628502 19841 hybrid_clock.cc:648] HybridClock initialized: now 1781859076628489 us; error 38 us; skew 500 ppm
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:16.631134 19841 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:51:16.631503 19841 webserver.cc:460] Webserver started at http://127.15.208.1:38057/ using document root <none> and password file <none>
I20260619 08:51:16.631680 19841 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:16.631719 19841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:16.631806 19841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:16.632848 19841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "792eb0e6753e4a2995701d46085b8d02"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "69114691f0177bb31f24bd5599cdbf71"
server_key_iv: "14978dfe0191ff48560e3cd1940616cd"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.633155 19841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "792eb0e6753e4a2995701d46085b8d02"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "69114691f0177bb31f24bd5599cdbf71"
server_key_iv: "14978dfe0191ff48560e3cd1940616cd"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.634366 19841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:16.635002 19856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.635171 19841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.635273 19841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "792eb0e6753e4a2995701d46085b8d02"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "69114691f0177bb31f24bd5599cdbf71"
server_key_iv: "14978dfe0191ff48560e3cd1940616cd"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.635345 19841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:16.648132 19841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:16.650537 19841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:16.650727 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:16.650964 19841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:16.651299 19841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:16.651355 19841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.651396 19841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:16.651424 19841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.657614 19841 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:33187
I20260619 08:51:16.657673 19969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:33187 every 8 connection(s)
I20260619 08:51:16.658013 19841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:16.662144 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 19841
I20260619 08:51:16.662257 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:16.662432 16192 external_mini_cluster.cc:1428] Setting key 433b6cbbda3d5199350e977fb3e7955b
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:16.668021 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:16.658747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:48705 (local address 127.15.208.62:35657)
0619 08:51:16.658833 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:16.658835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:16.659017 (+ 182us) server_negotiation.cc:407] Connection header received
0619 08:51:16.659888 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:16.659889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:16.659899 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:16.659958 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:16.660529 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.661152 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.662135 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.662462 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.662536 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:16.665113 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:16.665114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:16.665115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:16.665121 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:16.666558 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:16.667056 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:16.667057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:16.667057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:16.667108 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:16.667392 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:16.667393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:16.667393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:16.667487 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:16.667533 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:16.667885 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 08:51:16.667956 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 08:51:16.668346 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:16.658721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:35657 (local address 127.15.208.1:48705)
0619 08:51:16.658878 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:16.658919 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:16.659800 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:16.660021 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:16.660025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:16.660269 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:16.660458 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:16.660462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.661227 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.661228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:16.662043 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:16.662045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.662619 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.662621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:16.662757 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:16.663353 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:16.663361 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:16.665022 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:16.666643 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:16.666650 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:16.666653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:16.666975 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:16.667170 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:16.667170 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:16.667171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:16.667316 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:16.667553 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:16.667559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:16.667739 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 08:51:16.667870 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 08:51:16.668058 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260619 08:51:16.668921 19970 heartbeater.cc:347] Connected to a master server at 127.15.208.62:35657
I20260619 08:51:16.669059 19970 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:16.669238 19970 heartbeater.cc:511] Master 127.15.208.62:35657 requested a full tablet report, sending...
I20260619 08:51:16.669783 19779 ts_manager.cc:194] Registered new tserver with Master: 792eb0e6753e4a2995701d46085b8d02 (127.15.208.1:33187)
I20260619 08:51:16.670384 19779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:48705
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:51:16.687958 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:35657
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:16.767025 19975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:16.767266 19975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:16.767302 19975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:16.768476 19975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:16.768548 19975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:16.768589 19975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:16.770090 19975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:16.770360 19975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:16.770612 19975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:16.775414 19983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:16.775414 19980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:16.775455 19981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:16.775766 19975 server_base.cc:1034] running on GCE node
I20260619 08:51:16.775971 19975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:16.776162 19975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:16.777320 19975 hybrid_clock.cc:648] HybridClock initialized: now 1781859076777309 us; error 39 us; skew 500 ppm
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:16.779886 19975 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:51:16.780354 19975 webserver.cc:460] Webserver started at http://127.15.208.2:46317/ using document root <none> and password file <none>
I20260619 08:51:16.780580 19975 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:16.780624 19975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:16.780807 19975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:16.781865 19975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "d34dd8ca395e4c2e9534c6f97396f6c8"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "fd12c62b3b5cf6e229a206e107f03a78"
server_key_iv: "84c7a24472a74617f4e6c9c57e2d7f89"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.782235 19975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "d34dd8ca395e4c2e9534c6f97396f6c8"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "fd12c62b3b5cf6e229a206e107f03a78"
server_key_iv: "84c7a24472a74617f4e6c9c57e2d7f89"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.783666 19975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:16.784435 19990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.784601 19975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:16.784698 19975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "d34dd8ca395e4c2e9534c6f97396f6c8"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "fd12c62b3b5cf6e229a206e107f03a78"
server_key_iv: "84c7a24472a74617f4e6c9c57e2d7f89"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.784787 19975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:16.800585 19975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:16.802495 19975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:16.802668 19975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:16.802896 19975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:16.803228 19975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:16.803285 19975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.803326 19975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:16.803352 19975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.808887 19975 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:43795
I20260619 08:51:16.808979 20103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:43795 every 8 connection(s)
I20260619 08:51:16.809279 19975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:51:16.812661 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 19975
I20260619 08:51:16.812810 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:16.812987 16192 external_mini_cluster.cc:1428] Setting key d738ec011176dcc803882ccb2dda1052
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:16.818997 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:16.810027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:49287 (local address 127.15.208.62:35657)
0619 08:51:16.810086 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:16.810088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:16.810323 (+ 235us) server_negotiation.cc:407] Connection header received
0619 08:51:16.811217 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:16.811218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:16.811227 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:16.811286 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:16.811879 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.812367 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.813606 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.813811 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.813895 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:16.816213 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:16.816214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:16.816215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:16.816222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:16.817662 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:16.818090 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:16.818091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:16.818092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:16.818150 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:16.818390 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:16.818391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:16.818391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:16.818502 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:16.818576 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:16.818836 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 08:51:16.818909 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:51:16.819180 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:16.810001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:35657 (local address 127.15.208.2:49287)
0619 08:51:16.810164 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:16.810203 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:16.811128 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:16.811338 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:16.811341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:16.811594 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:16.811788 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:16.811791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.812443 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.812460 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:16.813498 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:16.813500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.813906 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.813907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:16.814055 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:16.814604 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:16.814612 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:16.816111 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:16.817762 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:16.817769 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:16.817771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:16.818007 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:16.818213 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:16.818214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:16.818214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:16.818324 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:16.818568 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:16.818574 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:16.818705 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 08:51:16.818803 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 08:51:16.818950 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260619 08:51:16.819681 20104 heartbeater.cc:347] Connected to a master server at 127.15.208.62:35657
I20260619 08:51:16.819758 20104 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:16.819938 20104 heartbeater.cc:511] Master 127.15.208.62:35657 requested a full tablet report, sending...
I20260619 08:51:16.820358 19779 ts_manager.cc:194] Registered new tserver with Master: d34dd8ca395e4c2e9534c6f97396f6c8 (127.15.208.2:43795)
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
I20260619 08:51:16.820887 19779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:49287
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:51:16.837533 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:35657
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:16.918912 20109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:16.919126 20109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:16.919150 20109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:16.920377 20109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:16.920420 20109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:16.920446 20109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:16.922060 20109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:16.922307 20109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:16.922538 20109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:16.927457 20114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:16.927464 20117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:16.927544 20115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:16.927670 20109 server_base.cc:1034] running on GCE node
I20260619 08:51:16.927879 20109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:16.928120 20109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:16.929286 20109 hybrid_clock.cc:648] HybridClock initialized: now 1781859076929285 us; error 44 us; skew 500 ppm
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:16.931764 20109 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:51:16.932245 20109 webserver.cc:460] Webserver started at http://127.15.208.3:46567/ using document root <none> and password file <none>
I20260619 08:51:16.932483 20109 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:16.932551 20109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:16.932766 20109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:16.933789 20109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "6e3cadeda06340dc9d9d08ef736b3873"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "abe1ab4866fd86737584313bf829329b"
server_key_iv: "70575463d472f8a1e283205f1a890740"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.934154 20109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "6e3cadeda06340dc9d9d08ef736b3873"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "abe1ab4866fd86737584313bf829329b"
server_key_iv: "70575463d472f8a1e283205f1a890740"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.935424 20109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:16.936111 20124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.936268 20109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:16.936368 20109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "6e3cadeda06340dc9d9d08ef736b3873"
format_stamp: "Formatted at 2026-06-19 08:51:16 on dist-test-slave-p0h9"
server_key: "abe1ab4866fd86737584313bf829329b"
server_key_iv: "70575463d472f8a1e283205f1a890740"
server_key_version: "encryptionkey@0"
I20260619 08:51:16.936447 20109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:16.956800 20109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:16.958721 20109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:16.958886 20109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:16.959115 20109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:16.959430 20109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:16.959486 20109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.959527 20109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:16.959555 20109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:16.965394 20109 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:42643
I20260619 08:51:16.965435 20237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:42643 every 8 connection(s)
I20260619 08:51:16.965750 20109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:16.972663 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 20109
I20260619 08:51:16.972805 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:16.973052 16192 external_mini_cluster.cc:1428] Setting key 81cb81624cd7ac595fae1b11d20318b1
I20260619 08:51:16.975553 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:16.966446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:50261 (local address 127.15.208.62:35657)
0619 08:51:16.966515 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:16.966517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:16.966723 (+ 206us) server_negotiation.cc:407] Connection header received
0619 08:51:16.967686 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:16.967687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:16.967699 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:16.967755 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:16.968384 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.968972 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.969979 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.970191 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.970268 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:16.972368 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:16.972369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:16.972369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:16.972375 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:16.974087 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:16.974530 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:16.974531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:16.974532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:16.974591 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:16.974904 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:16.974905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:16.974905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:16.975026 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:16.975106 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:16.975385 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 08:51:16.975463 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:51:16.975754 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:16.966429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:35657 (local address 127.15.208.3:50261)
0619 08:51:16.966602 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:16.966638 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:16.967556 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:16.967831 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:16.967835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:16.968085 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:16.968278 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:16.968282 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.969062 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.969063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:16.969875 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:16.969876 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:16.970273 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:16.970274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:16.970394 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:16.970816 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:16.970825 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:16.972279 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:16.974175 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:16.974181 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:16.974183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:16.974438 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:16.974698 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:16.974699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:16.974700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:16.974831 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:16.975096 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:16.975102 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:16.975248 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 08:51:16.975352 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 08:51:16.975510 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260619 08:51:16.976161 20238 heartbeater.cc:347] Connected to a master server at 127.15.208.62:35657
I20260619 08:51:16.976257 20238 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:16.976390 20238 heartbeater.cc:511] Master 127.15.208.62:35657 requested a full tablet report, sending...
I20260619 08:51:16.976747 19779 ts_manager.cc:194] Registered new tserver with Master: 6e3cadeda06340dc9d9d08ef736b3873 (127.15.208.3:42643)
I20260619 08:51:16.977098 19779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:50261
I20260619 08:51:16.984683 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:51:16 dist-test-slave-p0h9 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:51:17.007750 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:16.999486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47914 (local address 127.15.208.1:33187)
0619 08:51:16.999690 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:16.999695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:16.999706 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:51:16.999732 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:16.999734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:16.999748 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:16.999814 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:17.000315 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:17.001097 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:17.001868 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:17.002190 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:17.002244 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:17.004442 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:17.004450 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:17.004450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:17.004457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:17.006185 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:17.006675 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:17.006681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:17.006681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:17.006770 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:17.007035 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:17.007036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:17.007036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:17.007195 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:17.007371 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:17.007528 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 08:51:17.007619 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":119,"threads_started":1}
Jun 19 08:51:17 dist-test-slave-p0h9 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:17.015915 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:17.009219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36750 (local address 127.15.208.62:35657)
0619 08:51:17.009283 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:17.009286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:17.009295 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:17.009369 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:17.009370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:17.009379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:17.009436 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:17.009915 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:17.010404 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:17.011114 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:17.011308 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:17.011394 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:17.013549 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:17.013558 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:17.013559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:17.013566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:17.014772 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:17.015112 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:17.015114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:17.015114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:17.015172 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:17.015370 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:17.015371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:17.015371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:17.015526 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:17.015636 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:17.015737 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 08:51:17.015811 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 08:51:17.016197 19779 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36750
I20260619 08:51:17.016563 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 19841
I20260619 08:51:17.021193 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 19975
I20260619 08:51:17.025489 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 20109
I20260619 08:51:17.029711 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 19747
2026-06-19T08:51:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1890 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:17 dist-test-slave-p0h9 krb5kdc[20250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:17 dist-test-slave-p0h9 krb5kdc[20250](info): set up 2 sockets
Jun 19 08:51:17 dist-test-slave-p0h9 krb5kdc[20250](info): commencing operation
krb5kdc: starting...
W20260619 08:51:18.065287 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:18 dist-test-slave-p0h9 krb5kdc[20250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:18.187197 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:41511
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:41511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:18.271091 20263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:18.271303 20263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:18.271327 20263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:18.272585 20263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:18.272632 20263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:18.272646 20263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:18.272660 20263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:18.272671 20263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:18.274220 20263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:41511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:41511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:18.274498 20263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:18.274729 20263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:18.279860 20269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:18.279894 20268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:18.279852 20271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:18.280221 20263 server_base.cc:1034] running on GCE node
I20260619 08:51:18.280447 20263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:18.280823 20263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:18.281993 20263 hybrid_clock.cc:648] HybridClock initialized: now 1781859078281976 us; error 47 us; skew 500 ppm
Jun 19 08:51:18 dist-test-slave-p0h9 krb5kdc[20250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859078, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:18.289132 20263 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:18.289495 20263 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:18.289604 20263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:18.289757 20263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:18.290928 20263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "5071d40203b34821be1b6a24690c6e2f"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "730fd91d66cca6582a8695c170c86218"
server_key_iv: "6f1ed48655652d3f63011f63f737e6f5"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.291282 20263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "5071d40203b34821be1b6a24690c6e2f"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "730fd91d66cca6582a8695c170c86218"
server_key_iv: "6f1ed48655652d3f63011f63f737e6f5"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.292610 20263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:18.293426 20277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:18.293638 20263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:18.293745 20263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "5071d40203b34821be1b6a24690c6e2f"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "730fd91d66cca6582a8695c170c86218"
server_key_iv: "6f1ed48655652d3f63011f63f737e6f5"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.293819 20263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:18.305847 20263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:18.307971 20263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:18.308137 20263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:18.312261 20263 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:41511
I20260619 08:51:18.312284 20329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:41511 every 8 connection(s)
I20260619 08:51:18.312592 20263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:18.313572 20330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:18.315943 20330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f: Bootstrap starting.
I20260619 08:51:18.316562 20330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:18.316905 20330 log.cc:824] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:18.317634 20330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f: No bootstrap required, opened a new log
I20260619 08:51:18.319182 20330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5071d40203b34821be1b6a24690c6e2f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41511 } }
I20260619 08:51:18.319319 20330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:18.319342 20330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5071d40203b34821be1b6a24690c6e2f, State: Initialized, Role: FOLLOWER
I20260619 08:51:18.319504 20330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5071d40203b34821be1b6a24690c6e2f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41511 } }
I20260619 08:51:18.319581 20330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:18.319604 20330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:18.319676 20330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:18.320464 20330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5071d40203b34821be1b6a24690c6e2f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41511 } }
I20260619 08:51:18.320581 20330 leader_election.cc:302] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5071d40203b34821be1b6a24690c6e2f; no voters:
I20260619 08:51:18.320816 20330 leader_election.cc:288] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:18.320880 20333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:18.321100 20333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [term 1 LEADER]: Becoming Leader. State: Replica: 5071d40203b34821be1b6a24690c6e2f, State: Running, Role: LEADER
I20260619 08:51:18.321141 20330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:18.321290 20333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5071d40203b34821be1b6a24690c6e2f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41511 } }
I20260619 08:51:18.321699 20335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5071d40203b34821be1b6a24690c6e2f. Latest consensus state: current_term: 1 leader_uuid: "5071d40203b34821be1b6a24690c6e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5071d40203b34821be1b6a24690c6e2f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41511 } } }
I20260619 08:51:18.321810 20335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:18.321695 20334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5071d40203b34821be1b6a24690c6e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5071d40203b34821be1b6a24690c6e2f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41511 } } }
I20260619 08:51:18.321970 20334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:18.322072 20342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:18.322450 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 20263
I20260619 08:51:18.322583 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:18.322762 16192 external_mini_cluster.cc:1428] Setting key 5925f3374ce68c7200acbfeb5ae24832
I20260619 08:51:18.323333 20342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:18.325426 20342 catalog_manager.cc:1347] Generated new cluster ID: 9d82cc9a9e154f10a8c433cdf9d53a66
I20260619 08:51:18.325481 20342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:51:18 dist-test-slave-p0h9 krb5kdc[20250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:18.333632 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.323255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.15.208.62:41511)
0619 08:51:18.323602 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:18.323613 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:18.323650 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:51:18.324044 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:18.324054 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:18.324122 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:18.324297 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:18.325110 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.326180 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.326965 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.327510 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.327665 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.330099 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:18.330103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:18.330103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:18.330123 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:18.331813 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:18.332178 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:18.332181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:18.332182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:18.332283 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:18.332515 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:18.332516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:18.332517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:18.332977 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:18.333159 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:18.333260 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 08:51:18.333410 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":67,"threads_started":1}
I20260619 08:51:18.344180 20342 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:18.344959 20342 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:18.350888 20342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f: Generated new TSK 0
I20260619 08:51:18.351131 20342 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:51:18.381417 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:41511
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:18.464921 20356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:18.465148 20356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:18.465189 20356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:18.466349 20356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:18.466416 20356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:18.466459 20356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:18.467980 20356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:41511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:18.468233 20356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:18.468462 20356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:18.473321 20361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:18.473368 20362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:18.473321 20364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:18.473690 20356 server_base.cc:1034] running on GCE node
I20260619 08:51:18.473929 20356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:18.474181 20356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:18.475320 20356 hybrid_clock.cc:648] HybridClock initialized: now 1781859078475299 us; error 40 us; skew 500 ppm
Jun 19 08:51:18 dist-test-slave-p0h9 krb5kdc[20250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859078, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:18.478012 20356 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:51:18.478433 20356 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:18.478484 20356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:18.478650 20356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:18.479727 20356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "daf013e317264ffaa9df72a18c59d416"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "c7a7b27164d6a50546751f89cb5256c7"
server_key_iv: "e46a37f89261367863e864080e73bb17"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.480078 20356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "daf013e317264ffaa9df72a18c59d416"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "c7a7b27164d6a50546751f89cb5256c7"
server_key_iv: "e46a37f89261367863e864080e73bb17"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.481467 20356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:18.482250 20370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:18.482453 20356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:18.482568 20356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "daf013e317264ffaa9df72a18c59d416"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "c7a7b27164d6a50546751f89cb5256c7"
server_key_iv: "e46a37f89261367863e864080e73bb17"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.482637 20356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:18.494788 20356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:18.496901 20356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:18.497054 20356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:18.497316 20356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:18.497673 20356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:18.497726 20356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:18.497768 20356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:18.497795 20356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:18.503979 20356 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:33927
I20260619 08:51:18.504094 20483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:33927 every 8 connection(s)
I20260619 08:51:18.504343 20356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:18.506541 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 20356
I20260619 08:51:18.506685 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:18.506846 16192 external_mini_cluster.cc:1428] Setting key ed8d985b4efc8f2f6c5f35a3e1787ced
Jun 19 08:51:18 dist-test-slave-p0h9 krb5kdc[20250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859078, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:18.516227 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.505140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:52433 (local address 127.15.208.62:41511)
0619 08:51:18.505209 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:18.505211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:18.505522 (+ 311us) server_negotiation.cc:407] Connection header received
0619 08:51:18.506119 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:18.506120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:18.506139 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:18.506207 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:18.507546 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.508114 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.509800 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.510011 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.510118 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.512831 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:18.512833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:18.512833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:18.512841 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:18.514453 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:18.515058 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:18.515059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:18.515059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:18.515121 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:18.515475 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:18.515476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:18.515476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:18.515596 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:18.515655 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:18.516056 (+ 401us) server_negotiation.cc:299] Negotiation successful
0619 08:51:18.516147 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:51:18.516579 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.505121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:41511 (local address 127.15.208.1:52433)
0619 08:51:18.505329 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:18.505367 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:18.505994 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:18.506290 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:18.506294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:18.506612 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:18.507410 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.507424 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.508257 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.508259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.509694 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.509698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.510129 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.510131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.510266 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.510869 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:18.510878 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:18.512656 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:18.514551 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:18.514561 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:18.514563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:18.514949 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:18.515207 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:18.515208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:18.515209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:18.515367 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:18.515678 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:18.515686 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:18.515891 (+ 205us) client_negotiation.cc:769] Sending connection context
0619 08:51:18.516046 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 08:51:18.516263 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
I20260619 08:51:18.517386 20484 heartbeater.cc:347] Connected to a master server at 127.15.208.62:41511
I20260619 08:51:18.517499 20484 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:18.517647 20484 heartbeater.cc:511] Master 127.15.208.62:41511 requested a full tablet report, sending...
I20260619 08:51:18.518262 20294 ts_manager.cc:194] Registered new tserver with Master: daf013e317264ffaa9df72a18c59d416 (127.15.208.1:33927)
I20260619 08:51:18.518914 20294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:52433
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:51:18.535849 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:41511
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:18.621107 20489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:18.621322 20489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:18.621395 20489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:18.622702 20489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:18.622751 20489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:18.622781 20489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:18.624248 20489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:41511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:18.624459 20489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:18.624636 20489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:18.629518 20494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:18.629557 20497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:18.629590 20489 server_base.cc:1034] running on GCE node
W20260619 08:51:18.629554 20495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:18.629949 20489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:18.630204 20489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:18.631384 20489 hybrid_clock.cc:648] HybridClock initialized: now 1781859078631374 us; error 28 us; skew 500 ppm
Jun 19 08:51:18 dist-test-slave-p0h9 krb5kdc[20250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859078, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:18.633997 20489 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:51:18.634402 20489 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:18.634483 20489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:18.634644 20489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:18.635680 20489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "eac308e5b509472db335a2ca01948f33"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "16612af8e76d45cb2522892a0b743eba"
server_key_iv: "84cb231c69007b623614f200bd5bf81c"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.636059 20489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "eac308e5b509472db335a2ca01948f33"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "16612af8e76d45cb2522892a0b743eba"
server_key_iv: "84cb231c69007b623614f200bd5bf81c"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.637531 20489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:18.638288 20503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:18.638489 20489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:18.638589 20489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "eac308e5b509472db335a2ca01948f33"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "16612af8e76d45cb2522892a0b743eba"
server_key_iv: "84cb231c69007b623614f200bd5bf81c"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.638680 20489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:18.653208 20489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:18.654961 20489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:18.655126 20489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:18.655354 20489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:18.655663 20489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:18.655717 20489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:18.655751 20489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:18.655777 20489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:18.661725 20489 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:45393
I20260619 08:51:18.661767 20616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:45393 every 8 connection(s)
I20260619 08:51:18.662025 20489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:51:18 dist-test-slave-p0h9 krb5kdc[20250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859078, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:18.670760 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.662769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:41599 (local address 127.15.208.62:41511)
0619 08:51:18.662840 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:18.662842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:18.663010 (+ 168us) server_negotiation.cc:407] Connection header received
0619 08:51:18.663479 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:18.663480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:18.663489 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:18.663547 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:18.664456 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.665048 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.666009 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.666190 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.666241 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.668204 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:18.668205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:18.668205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:18.668212 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:18.669632 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:18.669998 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:18.669999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:18.669999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:18.670049 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:18.670256 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:18.670257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:18.670257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:18.670358 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:18.670405 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:18.670629 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 08:51:18.670695 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:51:18.670979 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 20489
I20260619 08:51:18.670996 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.662725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:41511 (local address 127.15.208.2:41599)
0619 08:51:18.662883 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:18.662925 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:18.663397 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:18.663618 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:18.663621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:18.663863 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:18.664370 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.664382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.665130 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.665131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.665934 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.665936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.666256 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.666257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.666353 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.666788 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:18.666796 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:18.668117 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:18.669699 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:18.669704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:18.669706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:18.669924 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:18.670109 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:18.670109 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:18.670109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:18.670197 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:18.670410 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:18.670413 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:18.670519 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 08:51:18.670603 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 08:51:18.670716 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260619 08:51:18.671093 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:18.671270 16192 external_mini_cluster.cc:1428] Setting key 3c4b00d2cd476fe10f08a300215e1490
I20260619 08:51:18.671697 20617 heartbeater.cc:347] Connected to a master server at 127.15.208.62:41511
I20260619 08:51:18.671792 20617 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:18.671945 20617 heartbeater.cc:511] Master 127.15.208.62:41511 requested a full tablet report, sending...
I20260619 08:51:18.672408 20294 ts_manager.cc:194] Registered new tserver with Master: eac308e5b509472db335a2ca01948f33 (127.15.208.2:45393)
I20260619 08:51:18.672869 20294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:41599
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:51:18.695878 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:41511
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:18.776463 20622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:18.776718 20622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:18.776770 20622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:18.777866 20622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:18.777902 20622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:18.777928 20622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:18.779461 20622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:41511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:18.779657 20622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:18.779831 20622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:18.784756 20630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:18.784744 20627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:18.784832 20628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:18.785000 20622 server_base.cc:1034] running on GCE node
I20260619 08:51:18.785245 20622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:18.785483 20622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:18.786621 20622 hybrid_clock.cc:648] HybridClock initialized: now 1781859078786597 us; error 40 us; skew 500 ppm
Jun 19 08:51:18 dist-test-slave-p0h9 krb5kdc[20250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859078, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:18.789145 20622 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:51:18.789431 20622 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:18.789474 20622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:18.789572 20622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:18.790702 20622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "2102a07b911141d98fcdab182f81fe28"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "7e572884748e43bbc0a4784d59db4532"
server_key_iv: "2f0e73a0ba78bbc979a318de0ad104d5"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.791004 20622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "2102a07b911141d98fcdab182f81fe28"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "7e572884748e43bbc0a4784d59db4532"
server_key_iv: "2f0e73a0ba78bbc979a318de0ad104d5"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.792539 20622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:18.793557 20636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:18.793848 20622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:18.793942 20622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "2102a07b911141d98fcdab182f81fe28"
format_stamp: "Formatted at 2026-06-19 08:51:18 on dist-test-slave-p0h9"
server_key: "7e572884748e43bbc0a4784d59db4532"
server_key_iv: "2f0e73a0ba78bbc979a318de0ad104d5"
server_key_version: "encryptionkey@0"
I20260619 08:51:18.794023 20622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:18.806129 20622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:18.807858 20622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:18.807968 20622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:18.808154 20622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:18.808460 20622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:18.808492 20622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:18.808539 20622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:18.808566 20622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:18.814841 20622 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:38781
I20260619 08:51:18.814927 20749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:38781 every 8 connection(s)
I20260619 08:51:18.815179 20622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:18.820850 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 20622
I20260619 08:51:18.820973 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:18.821228 16192 external_mini_cluster.cc:1428] Setting key 547d02ae5ea46991ea8e526773f16f18
Jun 19 08:51:18 dist-test-slave-p0h9 krb5kdc[20250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859078, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:18.824396 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.815917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:35803 (local address 127.15.208.62:41511)
0619 08:51:18.815986 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:18.815988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:18.816183 (+ 195us) server_negotiation.cc:407] Connection header received
0619 08:51:18.816636 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:18.816637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:18.816647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:18.816703 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:18.817688 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.818187 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.819225 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.819416 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.819497 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.821841 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:18.821843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:18.821843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:18.821870 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:18.823213 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:18.823615 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:18.823615 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:18.823616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:18.823665 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:18.823908 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:18.823909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:18.823909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:18.824006 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:18.824056 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:18.824275 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 08:51:18.824334 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:51:18.824610 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.815894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:41511 (local address 127.15.208.3:35803)
0619 08:51:18.816048 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:18.816098 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:18.816549 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:18.816823 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:18.816828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:18.817101 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:18.817580 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.817585 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.818290 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.818291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.819111 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.819113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.819506 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.819507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.819638 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.820120 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:18.820127 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:18.821769 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:18.823282 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:18.823287 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:18.823289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:18.823496 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:18.823719 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:18.823719 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:18.823720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:18.823852 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:18.824061 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:18.824064 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:18.824174 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 08:51:18.824255 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:51:18.824369 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260619 08:51:18.825086 20750 heartbeater.cc:347] Connected to a master server at 127.15.208.62:41511
I20260619 08:51:18.825181 20750 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:18.825294 20750 heartbeater.cc:511] Master 127.15.208.62:41511 requested a full tablet report, sending...
I20260619 08:51:18.825629 20294 ts_manager.cc:194] Registered new tserver with Master: 2102a07b911141d98fcdab182f81fe28 (127.15.208.3:38781)
I20260619 08:51:18.826047 20294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:35803
I20260619 08:51:18.833076 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:51:18.840153 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.834383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42340 (local address 127.15.208.62:41511)
0619 08:51:18.834454 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:18.834456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:18.834541 (+ 85us) server_negotiation.cc:407] Connection header received
0619 08:51:18.834622 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:18.834623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:18.834633 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:18.834690 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:18.835245 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.835731 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.836491 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.836685 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.836796 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.837846 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:18.837848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:18.837848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:18.837855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:18.838994 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:18.839335 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:18.839336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:18.839336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:18.839395 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:18.839615 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:18.839616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:18.839616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:18.839764 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:18.839869 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:18.839975 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:51:18.840051 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:51:18.841667 20294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:18.843319 20294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:18.849750 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.845542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:33927 (local address 127.0.0.1:35706)
0619 08:51:18.846144 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:18.846156 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:18.846204 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:18.846492 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:18.846493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:18.846495 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:18.846528 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.846530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.847729 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.847730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.848642 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.848644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.849432 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.849433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.849509 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.849525 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:51:18.849593 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 08:51:18.849646 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":53,"threads_started":1}
I20260619 08:51:18.849869 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.845507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:45393 (local address 127.0.0.1:39786)
0619 08:51:18.845915 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:18.845926 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:18.845976 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:18.846292 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:18.846293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:18.846295 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:18.846350 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.846352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.847293 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.847294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.848666 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.848669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.849609 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.849610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.849662 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.849665 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:18.849723 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:51:18.849757 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":33,"thread_start_us":71,"threads_started":1}
I20260619 08:51:18.849880 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.845774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39786 (local address 127.15.208.2:45393)
0619 08:51:18.846099 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:18.846104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:18.846112 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:18.846176 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:18.846179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:18.846194 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:18.846423 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:18.846497 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.847194 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.848812 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.849513 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.849605 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.849653 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:18.849738 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 08:51:18.849783 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":75,"threads_started":1}
I20260619 08:51:18.850088 20762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.845776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.15.208.1:33927)
0619 08:51:18.845952 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:18.845955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:18.846271 (+ 316us) server_negotiation.cc:407] Connection header received
0619 08:51:18.846317 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:18.846319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:18.846337 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:18.846406 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:18.846624 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.847634 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.848754 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.849346 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.849433 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.849505 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:18.849957 (+ 452us) server_negotiation.cc:299] Negotiation successful
0619 08:51:18.850015 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260619 08:51:18.850404 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.845498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:38781 (local address 127.0.0.1:48972)
0619 08:51:18.845687 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:18.845731 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:18.845800 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:18.846052 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:18.846054 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:18.846056 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:18.846112 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.846113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.847495 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.847496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.848667 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:18.848669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.849680 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.849682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:18.849764 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.849769 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:51:18.850266 (+ 497us) client_negotiation.cc:240] Negotiation successful
0619 08:51:18.850312 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":28,"thread_start_us":77,"threads_started":1}
I20260619 08:51:18.850636 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:18.845689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48972 (local address 127.15.208.3:38781)
0619 08:51:18.845874 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:18.845879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:18.845890 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:51:18.845933 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:18.845936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:18.845954 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:18.846040 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:18.846280 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.847393 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.848792 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:18.849513 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:18.849588 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:18.849644 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:18.850507 (+ 863us) server_negotiation.cc:299] Negotiation successful
0619 08:51:18.850550 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
I20260619 08:51:18.850620 20551 tablet_service.cc:1467] Processing CreateTablet for tablet 87e0a1557df34cc4a33c958359301ad6 (DEFAULT_TABLE table=test-table [id=8be3da94955245f58ece136053e88be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:18.850808 20418 tablet_service.cc:1467] Processing CreateTablet for tablet 87e0a1557df34cc4a33c958359301ad6 (DEFAULT_TABLE table=test-table [id=8be3da94955245f58ece136053e88be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:18.850914 20551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87e0a1557df34cc4a33c958359301ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:18.851056 20418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87e0a1557df34cc4a33c958359301ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:18.853523 20764 tablet_bootstrap.cc:492] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: Bootstrap starting.
I20260619 08:51:18.853992 20684 tablet_service.cc:1467] Processing CreateTablet for tablet 87e0a1557df34cc4a33c958359301ad6 (DEFAULT_TABLE table=test-table [id=8be3da94955245f58ece136053e88be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:18.854228 20764 tablet_bootstrap.cc:654] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:18.854321 20684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87e0a1557df34cc4a33c958359301ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:18.854812 20764 log.cc:824] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:18.855562 20764 tablet_bootstrap.cc:492] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: No bootstrap required, opened a new log
I20260619 08:51:18.855638 20764 ts_tablet_manager.cc:1397] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:51:18.855195 20765 tablet_bootstrap.cc:492] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416: Bootstrap starting.
I20260619 08:51:18.857273 20765 tablet_bootstrap.cc:654] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:18.857695 20767 tablet_bootstrap.cc:492] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: Bootstrap starting.
I20260619 08:51:18.857873 20765 log.cc:824] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:18.857785 20764 raft_consensus.cc:348] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 } } peers { permanent_uuid: "2102a07b911141d98fcdab182f81fe28" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38781 } } peers { permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 } }
I20260619 08:51:18.857971 20764 raft_consensus.cc:374] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:18.858022 20764 raft_consensus.cc:729] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac308e5b509472db335a2ca01948f33, State: Initialized, Role: FOLLOWER
I20260619 08:51:18.858151 20764 consensus_queue.cc:260] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 } } peers { permanent_uuid: "2102a07b911141d98fcdab182f81fe28" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38781 } } peers { permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 } }
I20260619 08:51:18.858314 20767 tablet_bootstrap.cc:654] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:18.858449 20764 ts_tablet_manager.cc:1428] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:51:18.858487 20617 heartbeater.cc:503] Master 127.15.208.62:41511 was elected leader, sending a full tablet report...
I20260619 08:51:18.858995 20767 log.cc:824] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:18.859148 20765 tablet_bootstrap.cc:492] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416: No bootstrap required, opened a new log
I20260619 08:51:18.859221 20765 ts_tablet_manager.cc:1397] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 08:51:18.859903 20767 tablet_bootstrap.cc:492] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: No bootstrap required, opened a new log
I20260619 08:51:18.859984 20767 ts_tablet_manager.cc:1397] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:51:18.860992 20765 raft_consensus.cc:348] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 } } peers { permanent_uuid: "2102a07b911141d98fcdab182f81fe28" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38781 } } peers { permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 } }
I20260619 08:51:18.861135 20765 raft_consensus.cc:374] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:18.861157 20765 raft_consensus.cc:729] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daf013e317264ffaa9df72a18c59d416, State: Initialized, Role: FOLLOWER
I20260619 08:51:18.861284 20765 consensus_queue.cc:260] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 } } peers { permanent_uuid: "2102a07b911141d98fcdab182f81fe28" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38781 } } peers { permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 } }
I20260619 08:51:18.861564 20765 ts_tablet_manager.cc:1428] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:51:18.861577 20484 heartbeater.cc:503] Master 127.15.208.62:41511 was elected leader, sending a full tablet report...
I20260619 08:51:18.861558 20767 raft_consensus.cc:348] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 } } peers { permanent_uuid: "2102a07b911141d98fcdab182f81fe28" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38781 } } peers { permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 } }
I20260619 08:51:18.861711 20767 raft_consensus.cc:374] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:18.861742 20767 raft_consensus.cc:729] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2102a07b911141d98fcdab182f81fe28, State: Initialized, Role: FOLLOWER
I20260619 08:51:18.861848 20767 consensus_queue.cc:260] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 } } peers { permanent_uuid: "2102a07b911141d98fcdab182f81fe28" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38781 } } peers { permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 } }
I20260619 08:51:18.862123 20767 ts_tablet_manager.cc:1428] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:51:18.862131 20750 heartbeater.cc:503] Master 127.15.208.62:41511 was elected leader, sending a full tablet report...
W20260619 08:51:18.912602 20618 tablet.cc:2369] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:51:19.005097 20485 tablet.cc:2369] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:51:19.065698 20751 tablet.cc:2369] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:51:19.145694 20772 raft_consensus.cc:482] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:19.145805 20772 raft_consensus.cc:504] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 } } peers { permanent_uuid: "2102a07b911141d98fcdab182f81fe28" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38781 } } peers { permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 } }
I20260619 08:51:19.146688 20772 leader_election.cc:288] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac308e5b509472db335a2ca01948f33 (127.15.208.2:45393), daf013e317264ffaa9df72a18c59d416 (127.15.208.1:33927)
I20260619 08:51:19.149979 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.146516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:37367 (local address 127.15.208.2:45393)
0619 08:51:19.146579 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.146582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.146592 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:19.146659 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.146660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.146668 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.146725 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:19.146922 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.147756 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.149131 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.149649 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.149719 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:19.149727 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.149866 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.149904 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:51:19.149977 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.146476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:45393 (local address 127.15.208.3:37367)
0619 08:51:19.146532 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.146544 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.146591 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.146779 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:19.146780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:19.146782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:19.146854 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:19.146856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.147834 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.147835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:19.149043 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:19.149046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.149733 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.149734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:19.149797 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:19.149800 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:19.149841 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 08:51:19.149870 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:51:19.150286 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.146855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:33927 (local address 127.15.208.3:44963)
0619 08:51:19.146994 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.147015 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.147055 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.147246 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:19.147247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:19.147249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:19.147309 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:19.147311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.148189 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.148190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:19.149311 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:19.149313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.150050 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.150052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:19.150122 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:19.150127 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:51:19.150175 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:51:19.150211 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260619 08:51:19.150295 20762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.147073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:44963 (local address 127.15.208.1:33927)
0619 08:51:19.147120 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.147122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.147132 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:19.147161 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.147162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.147171 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.147231 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:19.147361 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.148093 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.149389 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.149955 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.150026 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:19.150034 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.150194 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.150234 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:51:19.150298 20571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87e0a1557df34cc4a33c958359301ad6" candidate_uuid: "2102a07b911141d98fcdab182f81fe28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac308e5b509472db335a2ca01948f33" is_pre_election: true
I20260619 08:51:19.150470 20571 raft_consensus.cc:2454] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2102a07b911141d98fcdab182f81fe28 in term 0.
I20260619 08:51:19.150691 20424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87e0a1557df34cc4a33c958359301ad6" candidate_uuid: "2102a07b911141d98fcdab182f81fe28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daf013e317264ffaa9df72a18c59d416" is_pre_election: true
I20260619 08:51:19.150736 20637 leader_election.cc:302] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2102a07b911141d98fcdab182f81fe28, eac308e5b509472db335a2ca01948f33; no voters:
I20260619 08:51:19.150839 20424 raft_consensus.cc:2454] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2102a07b911141d98fcdab182f81fe28 in term 0.
I20260619 08:51:19.150857 20772 raft_consensus.cc:2781] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:19.150926 20772 raft_consensus.cc:482] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:19.150956 20772 raft_consensus.cc:3037] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:19.151861 20772 raft_consensus.cc:504] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 } } peers { permanent_uuid: "2102a07b911141d98fcdab182f81fe28" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38781 } } peers { permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 } }
I20260619 08:51:19.152014 20772 leader_election.cc:288] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [CANDIDATE]: Term 1 election: Requested vote from peers eac308e5b509472db335a2ca01948f33 (127.15.208.2:45393), daf013e317264ffaa9df72a18c59d416 (127.15.208.1:33927)
I20260619 08:51:19.152171 20424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87e0a1557df34cc4a33c958359301ad6" candidate_uuid: "2102a07b911141d98fcdab182f81fe28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daf013e317264ffaa9df72a18c59d416"
I20260619 08:51:19.152275 20424 raft_consensus.cc:3037] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:19.153369 20424 raft_consensus.cc:2454] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2102a07b911141d98fcdab182f81fe28 in term 1.
I20260619 08:51:19.153491 20571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87e0a1557df34cc4a33c958359301ad6" candidate_uuid: "2102a07b911141d98fcdab182f81fe28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac308e5b509472db335a2ca01948f33"
I20260619 08:51:19.153534 20639 leader_election.cc:302] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2102a07b911141d98fcdab182f81fe28, daf013e317264ffaa9df72a18c59d416; no voters:
I20260619 08:51:19.153576 20571 raft_consensus.cc:3037] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:19.153618 20772 raft_consensus.cc:2781] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:19.153757 20772 raft_consensus.cc:686] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 1 LEADER]: Becoming Leader. State: Replica: 2102a07b911141d98fcdab182f81fe28, State: Running, Role: LEADER
I20260619 08:51:19.153856 20772 consensus_queue.cc:237] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 } } peers { permanent_uuid: "2102a07b911141d98fcdab182f81fe28" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38781 } } peers { permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 } }
I20260619 08:51:19.154482 20571 raft_consensus.cc:2454] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2102a07b911141d98fcdab182f81fe28 in term 1.
I20260619 08:51:19.154819 20294 catalog_manager.cc:5495] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2102a07b911141d98fcdab182f81fe28 (127.15.208.3). New cstate: current_term: 1 leader_uuid: "2102a07b911141d98fcdab182f81fe28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2102a07b911141d98fcdab182f81fe28" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:19.204028 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.201929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.15.208.3:38781)
0619 08:51:19.201985 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.201987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.201994 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:51:19.202031 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.202032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.202048 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.202115 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:19.202231 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.202684 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.203395 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.203592 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.203644 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:19.203766 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:19.203863 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:19.203926 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.203955 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.203972 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:51:19.205955 20571 raft_consensus.cc:1261] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33 [term 1 FOLLOWER]: Refusing update from remote peer 2102a07b911141d98fcdab182f81fe28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:19.205961 20424 raft_consensus.cc:1261] T 87e0a1557df34cc4a33c958359301ad6 P daf013e317264ffaa9df72a18c59d416 [term 1 FOLLOWER]: Refusing update from remote peer 2102a07b911141d98fcdab182f81fe28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:19.206305 20772 consensus_queue.cc:1035] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac308e5b509472db335a2ca01948f33" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:19.206434 20774 consensus_queue.cc:1035] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "daf013e317264ffaa9df72a18c59d416" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:19.209730 20779 mvcc.cc:205] Tried to move back new op lower bound from 7298494788424462336 to 7298494788214968320. Current Snapshot: MvccSnapshot[applied={T|T < 7298494788424462336}]
I20260619 08:51:19.209748 20778 mvcc.cc:205] Tried to move back new op lower bound from 7298494788424462336 to 7298494788214968320. Current Snapshot: MvccSnapshot[applied={T|T < 7298494788424462336 or (T in {7298494788424462336})}]
I20260619 08:51:19.211047 20293 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:19.211151 20293 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:19.211906 20293 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5071d40203b34821be1b6a24690c6e2f: Sending DeleteTablet for 3 replicas of tablet 87e0a1557df34cc4a33c958359301ad6
I20260619 08:51:19.212270 20551 tablet_service.cc:1514] Processing DeleteTablet for tablet 87e0a1557df34cc4a33c958359301ad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:19 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:39786
I20260619 08:51:19.212281 20418 tablet_service.cc:1514] Processing DeleteTablet for tablet 87e0a1557df34cc4a33c958359301ad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:19 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:35706
I20260619 08:51:19.212486 20684 tablet_service.cc:1514] Processing DeleteTablet for tablet 87e0a1557df34cc4a33c958359301ad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:19 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:48972
I20260619 08:51:19.212694 20789 tablet_replica.cc:330] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: stopping tablet replica
I20260619 08:51:19.212795 20788 tablet_replica.cc:330] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: stopping tablet replica
I20260619 08:51:19.212858 20789 raft_consensus.cc:2229] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:19.212877 20788 raft_consensus.cc:2229] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:19.212975 20789 raft_consensus.cc:2258] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:19.212981 20788 raft_consensus.cc:2258] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:19.213157 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 20356
I20260619 08:51:19.213407 20788 ts_tablet_manager.cc:1905] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:19.214033 20789 ts_tablet_manager.cc:1905] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:19.215389 20788 ts_tablet_manager.cc:1918] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:19.215467 20788 log.cc:1196] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/87e0a1557df34cc4a33c958359301ad6
I20260619 08:51:19.215726 20788 ts_tablet_manager.cc:1939] T 87e0a1557df34cc4a33c958359301ad6 P eac308e5b509472db335a2ca01948f33: Deleting consensus metadata
I20260619 08:51:19.216142 20278 catalog_manager.cc:4841] TS eac308e5b509472db335a2ca01948f33 (127.15.208.2:45393): tablet 87e0a1557df34cc4a33c958359301ad6 (table test-table [id=8be3da94955245f58ece136053e88be1]) successfully deleted
I20260619 08:51:19.216274 20789 ts_tablet_manager.cc:1918] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:19.216347 20789 log.cc:1196] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/87e0a1557df34cc4a33c958359301ad6
I20260619 08:51:19.216602 20789 ts_tablet_manager.cc:1939] T 87e0a1557df34cc4a33c958359301ad6 P 2102a07b911141d98fcdab182f81fe28: Deleting consensus metadata
I20260619 08:51:19.217072 20279 catalog_manager.cc:4841] TS 2102a07b911141d98fcdab182f81fe28 (127.15.208.3:38781): tablet 87e0a1557df34cc4a33c958359301ad6 (table test-table [id=8be3da94955245f58ece136053e88be1]) successfully deleted
W20260619 08:51:19.220273 20280 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:33927 (error 108)
W20260619 08:51:19.220892 20760 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:19.220662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:33927 (local address 127.0.0.1:35716)
0619 08:51:19.220699 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.220711 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.220773 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.220838 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:33927: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":17}
W20260619 08:51:19.221010 20280 catalog_manager.cc:4584] TS daf013e317264ffaa9df72a18c59d416 (127.15.208.1:33927): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 87e0a1557df34cc4a33c958359301ad6: Network error: Client connection negotiation failed: client connection to 127.15.208.1:33927: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260619 08:51:19.221167 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 20489
I20260619 08:51:19.226697 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 20622
I20260619 08:51:19.231781 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 20263
2026-06-19T08:51:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2203 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T08:51:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:19Z Disabled control of system clock
I20260619 08:51:19.259443 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:33005
--webserver_interface=127.15.208.62
--webserver_port=0
--builtin_ntp_servers=127.15.208.20:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:33005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 08:51:19.340875 20794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:19.341094 20794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:19.341116 20794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:19.342296 20794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:19.342342 20794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:19.342355 20794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:19.342398 20794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:19.342425 20794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:19.343784 20794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:33005
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:33005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:19.344061 20794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:19.344379 20794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:51:19.349386 20794 server_base.cc:1034] running on GCE node
W20260619 08:51:19.349367 20799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:19.349334 20800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:19.349329 20802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:19.349895 20794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:19.350214 20794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:19.351398 20794 hybrid_clock.cc:648] HybridClock initialized: now 1781859079351388 us; error 37 us; skew 500 ppm
I20260619 08:51:19.352703 20794 webserver.cc:460] Webserver started at http://127.15.208.62:35533/ using document root <none> and password file <none>
I20260619 08:51:19.352998 20794 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:19.353086 20794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:19.353231 20794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:19.354331 20794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "4426725a526f42a29e40ac47ef3a75f4"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "436fceee23d71a7e15db1970ecfbf0ce"
server_key_iv: "e2420fe809d964c428d39eb766c56936"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.354691 20794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "4426725a526f42a29e40ac47ef3a75f4"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "436fceee23d71a7e15db1970ecfbf0ce"
server_key_iv: "e2420fe809d964c428d39eb766c56936"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.355999 20794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:19.356716 20808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.356981 20794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:19.357087 20794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "4426725a526f42a29e40ac47ef3a75f4"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "436fceee23d71a7e15db1970ecfbf0ce"
server_key_iv: "e2420fe809d964c428d39eb766c56936"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.357169 20794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:19.360476 20794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:19.360792 20794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:19.360930 20794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:19.364962 20794 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:33005
I20260619 08:51:19.365001 20860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:33005 every 8 connection(s)
I20260619 08:51:19.365367 20794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:19.365988 20861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:19.368690 20861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4: Bootstrap starting.
I20260619 08:51:19.369294 20861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:19.369606 20861 log.cc:824] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:19.370250 20861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4: No bootstrap required, opened a new log
I20260619 08:51:19.371933 20861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4426725a526f42a29e40ac47ef3a75f4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33005 } }
I20260619 08:51:19.372076 20861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:19.372098 20861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4426725a526f42a29e40ac47ef3a75f4, State: Initialized, Role: FOLLOWER
I20260619 08:51:19.372257 20861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4426725a526f42a29e40ac47ef3a75f4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33005 } }
I20260619 08:51:19.372321 20861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:19.372342 20861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:19.372406 20861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:19.373039 20861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4426725a526f42a29e40ac47ef3a75f4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33005 } }
I20260619 08:51:19.373148 20861 leader_election.cc:302] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4426725a526f42a29e40ac47ef3a75f4; no voters:
I20260619 08:51:19.373323 20861 leader_election.cc:288] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:19.373375 20864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:19.373586 20864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [term 1 LEADER]: Becoming Leader. State: Replica: 4426725a526f42a29e40ac47ef3a75f4, State: Running, Role: LEADER
I20260619 08:51:19.373669 20861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:19.373679 20864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4426725a526f42a29e40ac47ef3a75f4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33005 } }
I20260619 08:51:19.374064 20865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4426725a526f42a29e40ac47ef3a75f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4426725a526f42a29e40ac47ef3a75f4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33005 } } }
I20260619 08:51:19.374151 20865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:19.374284 20866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4426725a526f42a29e40ac47ef3a75f4. Latest consensus state: current_term: 1 leader_uuid: "4426725a526f42a29e40ac47ef3a75f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4426725a526f42a29e40ac47ef3a75f4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33005 } } }
I20260619 08:51:19.374354 20866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:19.374394 20872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:19.374519 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 20794
I20260619 08:51:19.374675 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:19.374871 16192 external_mini_cluster.cc:1428] Setting key 6945e4c409fd30543ff1335ac6d1dae4
I20260619 08:51:19.374891 20872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:19.377422 20872 catalog_manager.cc:1347] Generated new cluster ID: 6dc64e8bc32a45d08dcd1be877e4fc58
I20260619 08:51:19.377480 20872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:19.377916 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.375382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34226 (local address 127.15.208.62:33005)
0619 08:51:19.376345 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.376353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.376386 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:51:19.376497 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.376504 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.376547 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.376683 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.377476 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.377480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.377481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.377486 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.377537 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.377551 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.377606 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.377680 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.377740 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":80,"threads_started":1}
I20260619 08:51:19.388011 20872 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:19.388219 20872 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:19.400147 20872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4: Generated new TSK 0
I20260619 08:51:19.400367 20872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:51:19.403689 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33005
--builtin_ntp_servers=127.15.208.20:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 08:51:19.492363 20885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:19.492583 20885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:19.492607 20885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:19.493824 20885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:19.493870 20885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:19.493896 20885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:19.495260 20885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:19.495518 20885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:19.495834 20885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:19.501145 20891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:19.501128 20890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:19.501345 20885 server_base.cc:1034] running on GCE node
W20260619 08:51:19.501467 20893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:19.501685 20885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:19.501958 20885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:19.503119 20885 hybrid_clock.cc:648] HybridClock initialized: now 1781859079503099 us; error 38 us; skew 500 ppm
I20260619 08:51:19.504429 20885 webserver.cc:460] Webserver started at http://127.15.208.1:36799/ using document root <none> and password file <none>
I20260619 08:51:19.504680 20885 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:19.504767 20885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:19.504930 20885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:19.506000 20885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "88269a2cee184e7eb9c457c6b406bed8"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "5748ad964f70c54551fe6f8582c26b15"
server_key_iv: "80a46ac6bb6f7909c4e30b1f73f018ec"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.506362 20885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "88269a2cee184e7eb9c457c6b406bed8"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "5748ad964f70c54551fe6f8582c26b15"
server_key_iv: "80a46ac6bb6f7909c4e30b1f73f018ec"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.507686 20885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:19.508452 20899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.508627 20885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:19.508741 20885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "88269a2cee184e7eb9c457c6b406bed8"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "5748ad964f70c54551fe6f8582c26b15"
server_key_iv: "80a46ac6bb6f7909c4e30b1f73f018ec"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.508831 20885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:19.512013 20885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:19.512300 20885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:19.512451 20885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:19.512692 20885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:19.513055 20885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:19.513115 20885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.513156 20885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:19.513183 20885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.518563 20885 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:45415
I20260619 08:51:19.518664 21012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:45415 every 8 connection(s)
I20260619 08:51:19.518978 20885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:19.521788 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.519690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:34139 (local address 127.15.208.62:33005)
0619 08:51:19.519754 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.519758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.519963 (+ 205us) server_negotiation.cc:407] Connection header received
0619 08:51:19.520430 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.520431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.520436 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.520492 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.521600 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.521601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.521601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.521605 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.521618 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.521622 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.521670 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.521726 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.521739 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:51:19.521952 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.519654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:33005 (local address 127.15.208.1:34139)
0619 08:51:19.519835 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.519874 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.520349 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.520551 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:19.520555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:19.520564 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:19.521461 (+ 897us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:19.521461 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:19.521488 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:19.521516 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:19.521667 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:19.521669 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:19.521672 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:19.521721 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:51:19.521765 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260619 08:51:19.522478 21013 heartbeater.cc:347] Connected to a master server at 127.15.208.62:33005
I20260619 08:51:19.522569 21013 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:19.522678 21013 heartbeater.cc:511] Master 127.15.208.62:33005 requested a full tablet report, sending...
I20260619 08:51:19.523136 20825 ts_manager.cc:194] Registered new tserver with Master: 88269a2cee184e7eb9c457c6b406bed8 (127.15.208.1:45415)
I20260619 08:51:19.529063 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 20885
I20260619 08:51:19.529145 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:19.529280 16192 external_mini_cluster.cc:1428] Setting key 7d6287bc655aef6f7bd445afa8e8413f
I20260619 08:51:19.530150 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33005
--builtin_ntp_servers=127.15.208.20:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 08:51:19.610173 21016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:19.610469 21016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:19.610517 21016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:19.611912 21016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:19.611981 21016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:19.612022 21016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:19.613512 21016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:19.613817 21016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:19.614156 21016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:19.619145 21024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:19.619194 21022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:19.619163 21021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:19.619371 21016 server_base.cc:1034] running on GCE node
I20260619 08:51:19.619585 21016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:19.619818 21016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:19.620978 21016 hybrid_clock.cc:648] HybridClock initialized: now 1781859079620972 us; error 36 us; skew 500 ppm
I20260619 08:51:19.622318 21016 webserver.cc:460] Webserver started at http://127.15.208.2:36765/ using document root <none> and password file <none>
I20260619 08:51:19.622601 21016 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:19.622676 21016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:19.622826 21016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:19.623904 21016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "d74db241c01b4b0f817bf6c2183d85e1"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "e25cb7ad05d6fdd068110b318f1a4a59"
server_key_iv: "b2c9462d98c570691778a7f245d5605a"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.624260 21016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "d74db241c01b4b0f817bf6c2183d85e1"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "e25cb7ad05d6fdd068110b318f1a4a59"
server_key_iv: "b2c9462d98c570691778a7f245d5605a"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.625741 21016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:19.626631 21030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.626803 21016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:19.626914 21016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "d74db241c01b4b0f817bf6c2183d85e1"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "e25cb7ad05d6fdd068110b318f1a4a59"
server_key_iv: "b2c9462d98c570691778a7f245d5605a"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.626999 21016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:19.630647 21016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:19.630931 21016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:19.631080 21016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:19.631290 21016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:19.631613 21016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:19.631680 21016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.631721 21016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:19.631752 21016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.638051 21016 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:46853
I20260619 08:51:19.638089 21143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:46853 every 8 connection(s)
I20260619 08:51:19.638440 21016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:51:19.641433 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.639232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:60837 (local address 127.15.208.62:33005)
0619 08:51:19.639285 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.639288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.639536 (+ 248us) server_negotiation.cc:407] Connection header received
0619 08:51:19.640012 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.640013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.640018 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.640081 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.641237 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.641239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.641239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.641242 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.641257 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.641260 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.641306 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.641367 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.641379 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:51:19.641614 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.639222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:33005 (local address 127.15.208.2:60837)
0619 08:51:19.639412 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.639445 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.639893 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.640149 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:19.640153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:19.640161 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:19.641091 (+ 930us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:19.641092 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:19.641119 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:19.641145 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:19.641308 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:19.641310 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:19.641313 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:19.641361 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:51:19.641407 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
I20260619 08:51:19.642079 21144 heartbeater.cc:347] Connected to a master server at 127.15.208.62:33005
I20260619 08:51:19.642161 21144 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:19.642266 21144 heartbeater.cc:511] Master 127.15.208.62:33005 requested a full tablet report, sending...
I20260619 08:51:19.642601 20825 ts_manager.cc:194] Registered new tserver with Master: d74db241c01b4b0f817bf6c2183d85e1 (127.15.208.2:46853)
I20260619 08:51:19.644603 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 21016
I20260619 08:51:19.644688 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:19.644872 16192 external_mini_cluster.cc:1428] Setting key c8769d872ffcd7fa423b211ba5306073
I20260619 08:51:19.645769 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33005
--builtin_ntp_servers=127.15.208.20:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 08:51:19.726102 21147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:19.726302 21147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:19.726325 21147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:19.727397 21147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:19.727429 21147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:19.727452 21147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:19.728708 21147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:19.728956 21147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:19.729189 21147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:19.734071 21153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:19.734107 21152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:19.734107 21155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:19.734417 21147 server_base.cc:1034] running on GCE node
I20260619 08:51:19.734637 21147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:19.734822 21147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:19.735955 21147 hybrid_clock.cc:648] HybridClock initialized: now 1781859079735946 us; error 31 us; skew 500 ppm
I20260619 08:51:19.737232 21147 webserver.cc:460] Webserver started at http://127.15.208.3:41621/ using document root <none> and password file <none>
I20260619 08:51:19.737488 21147 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:19.737533 21147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:19.737707 21147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:19.738780 21147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "c307afd096f7430286ed54e1152c0e62"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "61d9085cd68cc63edf08dacae5ec5ed0"
server_key_iv: "33aceb010028959964622e96999e1016"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.739131 21147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "c307afd096f7430286ed54e1152c0e62"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "61d9085cd68cc63edf08dacae5ec5ed0"
server_key_iv: "33aceb010028959964622e96999e1016"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.740487 21147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:51:19.741256 21161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.741428 21147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.741524 21147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "c307afd096f7430286ed54e1152c0e62"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "61d9085cd68cc63edf08dacae5ec5ed0"
server_key_iv: "33aceb010028959964622e96999e1016"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.741611 21147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:19.744865 21147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:19.745126 21147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:19.745282 21147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:19.745513 21147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:19.745824 21147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:19.745877 21147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.745918 21147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:19.745944 21147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.751602 21147 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:35935
I20260619 08:51:19.751648 21274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:35935 every 8 connection(s)
I20260619 08:51:19.751940 21147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:19.754684 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.752642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:51917 (local address 127.15.208.62:33005)
0619 08:51:19.752709 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.752711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.752951 (+ 240us) server_negotiation.cc:407] Connection header received
0619 08:51:19.753375 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.753376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.753381 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.753433 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.754467 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.754468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.754469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.754472 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.754485 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.754489 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.754576 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.754623 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.754635 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:51:19.754860 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.752594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:33005 (local address 127.15.208.3:51917)
0619 08:51:19.752835 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.752867 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.753285 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.753480 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:19.753483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:19.753490 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:19.754336 (+ 846us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:19.754336 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:19.754361 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:19.754386 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:19.754575 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:19.754577 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:19.754579 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:19.754628 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:51:19.754670 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":134,"threads_started":1}
I20260619 08:51:19.755357 21275 heartbeater.cc:347] Connected to a master server at 127.15.208.62:33005
I20260619 08:51:19.755429 21275 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:19.755602 21275 heartbeater.cc:511] Master 127.15.208.62:33005 requested a full tablet report, sending...
I20260619 08:51:19.755970 20825 ts_manager.cc:194] Registered new tserver with Master: c307afd096f7430286ed54e1152c0e62 (127.15.208.3:35935)
I20260619 08:51:19.760391 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 21147
I20260619 08:51:19.760521 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:19.760779 16192 external_mini_cluster.cc:1428] Setting key 4bf32276fca6ec14f522f0e0cfc674fa
I20260619 08:51:19.761813 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:51:19.765777 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.763343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.15.208.62:33005)
0619 08:51:19.764870 (+ 1527us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.764872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.764882 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:19.764906 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.764907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.764911 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.764974 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.765553 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.765554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.765554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.765557 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.765573 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.765577 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.765617 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.765655 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.765668 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1498}
I20260619 08:51:19.767271 20825 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:19.768714 20825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:19.772575 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.771213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.15.208.2:46853)
0619 08:51:19.771389 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.771393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.771437 (+ 44us) server_negotiation.cc:407] Connection header received
0619 08:51:19.771552 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.771556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.771565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.771637 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.772269 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.772276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.772277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.772281 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.772316 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.772341 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.772402 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.772499 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.772512 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260619 08:51:19.772580 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.771024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:46853 (local address 127.0.0.1:42790)
0619 08:51:19.771386 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.771396 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.771442 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.771906 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:19.771907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:19.771910 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:19.772173 (+ 263us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:19.772174 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:19.772179 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:19.772187 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:19.772399 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:19.772405 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:19.772409 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:19.772486 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:51:19.772500 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":61,"threads_started":1}
I20260619 08:51:19.772902 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.770830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:45415 (local address 127.0.0.1:39314)
0619 08:51:19.771006 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.771048 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.771118 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.771694 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:19.771695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:19.771697 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:19.772135 (+ 438us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:19.772136 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:19.772153 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:19.772177 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:19.772781 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:19.772782 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:19.772783 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:51:19.772834 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 08:51:19.772847 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":57,"threads_started":1}
I20260619 08:51:19.772956 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.770912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39314 (local address 127.15.208.1:45415)
0619 08:51:19.771223 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.771228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.771237 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:19.771269 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.771270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.771278 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.771369 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.772269 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.772277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.772277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.772281 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.772316 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.772341 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.772403 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.772852 (+ 449us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.772868 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":107,"threads_started":1}
I20260619 08:51:19.773288 21078 tablet_service.cc:1467] Processing CreateTablet for tablet bd4f8bede2b94eb3988510a576ec7edb (DEFAULT_TABLE table=test-table [id=7f9707a886dc4f3798144ae4ec9a12fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:19.773669 21078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd4f8bede2b94eb3988510a576ec7edb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:19.773581 20947 tablet_service.cc:1467] Processing CreateTablet for tablet bd4f8bede2b94eb3988510a576ec7edb (DEFAULT_TABLE table=test-table [id=7f9707a886dc4f3798144ae4ec9a12fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:19.773923 20947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd4f8bede2b94eb3988510a576ec7edb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:19.776428 21289 tablet_bootstrap.cc:492] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: Bootstrap starting.
I20260619 08:51:19.776587 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.770858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:35935 (local address 127.0.0.1:57904)
0619 08:51:19.771126 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.771137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.771182 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.771459 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:19.771465 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:19.771472 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:19.772336 (+ 864us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:19.772337 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:19.772344 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:19.772356 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:19.776454 (+ 4098us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:19.776455 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:19.776456 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:51:19.776509 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:51:19.776524 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":93,"threads_started":1}
I20260619 08:51:19.776827 21287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.771012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57904 (local address 127.15.208.3:35935)
0619 08:51:19.771306 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.771311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.771320 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:19.771344 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.771346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.771356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.772246 (+ 890us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.772630 (+ 384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.772633 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.772634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.772637 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.772668 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.772682 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.772775 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.776681 (+ 3906us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.776700 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":92,"threads_started":1}
I20260619 08:51:19.776896 21290 tablet_bootstrap.cc:492] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8: Bootstrap starting.
I20260619 08:51:19.777171 21289 tablet_bootstrap.cc:654] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:19.777423 21209 tablet_service.cc:1467] Processing CreateTablet for tablet bd4f8bede2b94eb3988510a576ec7edb (DEFAULT_TABLE table=test-table [id=7f9707a886dc4f3798144ae4ec9a12fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:19.777498 21289 log.cc:824] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:19.777535 21290 tablet_bootstrap.cc:654] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:19.777756 21209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd4f8bede2b94eb3988510a576ec7edb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:19.777866 21290 log.cc:824] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:19.778261 21289 tablet_bootstrap.cc:492] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: No bootstrap required, opened a new log
I20260619 08:51:19.778343 21289 ts_tablet_manager.cc:1397] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 08:51:19.778582 21290 tablet_bootstrap.cc:492] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8: No bootstrap required, opened a new log
I20260619 08:51:19.778661 21290 ts_tablet_manager.cc:1397] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:19.780553 21290 raft_consensus.cc:348] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 } } peers { permanent_uuid: "d74db241c01b4b0f817bf6c2183d85e1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46853 } } peers { permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 } }
I20260619 08:51:19.780705 21290 raft_consensus.cc:374] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:19.780774 21290 raft_consensus.cc:729] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88269a2cee184e7eb9c457c6b406bed8, State: Initialized, Role: FOLLOWER
I20260619 08:51:19.781108 21293 tablet_bootstrap.cc:492] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: Bootstrap starting.
I20260619 08:51:19.781075 21290 consensus_queue.cc:260] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 } } peers { permanent_uuid: "d74db241c01b4b0f817bf6c2183d85e1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46853 } } peers { permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 } }
I20260619 08:51:19.781347 21290 ts_tablet_manager.cc:1428] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:51:19.781255 21289 raft_consensus.cc:348] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 } } peers { permanent_uuid: "d74db241c01b4b0f817bf6c2183d85e1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46853 } } peers { permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 } }
I20260619 08:51:19.781423 21289 raft_consensus.cc:374] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:19.781466 21289 raft_consensus.cc:729] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74db241c01b4b0f817bf6c2183d85e1, State: Initialized, Role: FOLLOWER
I20260619 08:51:19.781497 21013 heartbeater.cc:503] Master 127.15.208.62:33005 was elected leader, sending a full tablet report...
I20260619 08:51:19.781584 21289 consensus_queue.cc:260] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 } } peers { permanent_uuid: "d74db241c01b4b0f817bf6c2183d85e1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46853 } } peers { permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 } }
I20260619 08:51:19.781744 21293 tablet_bootstrap.cc:654] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:19.782032 21293 log.cc:824] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:19.782225 21289 ts_tablet_manager.cc:1428] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260619 08:51:19.782334 21144 heartbeater.cc:503] Master 127.15.208.62:33005 was elected leader, sending a full tablet report...
I20260619 08:51:19.783202 21293 tablet_bootstrap.cc:492] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: No bootstrap required, opened a new log
I20260619 08:51:19.783281 21293 ts_tablet_manager.cc:1397] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:19.785074 21293 raft_consensus.cc:348] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 } } peers { permanent_uuid: "d74db241c01b4b0f817bf6c2183d85e1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46853 } } peers { permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 } }
I20260619 08:51:19.785209 21293 raft_consensus.cc:374] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:19.785251 21293 raft_consensus.cc:729] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c307afd096f7430286ed54e1152c0e62, State: Initialized, Role: FOLLOWER
I20260619 08:51:19.785357 21293 consensus_queue.cc:260] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 } } peers { permanent_uuid: "d74db241c01b4b0f817bf6c2183d85e1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46853 } } peers { permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 } }
I20260619 08:51:19.785580 21293 ts_tablet_manager.cc:1428] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:51:19.785660 21275 heartbeater.cc:503] Master 127.15.208.62:33005 was elected leader, sending a full tablet report...
I20260619 08:51:19.786835 21295 raft_consensus.cc:482] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:19.786895 21295 raft_consensus.cc:504] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 } } peers { permanent_uuid: "d74db241c01b4b0f817bf6c2183d85e1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46853 } } peers { permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 } }
I20260619 08:51:19.787117 21295 leader_election.cc:288] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88269a2cee184e7eb9c457c6b406bed8 (127.15.208.1:45415), c307afd096f7430286ed54e1152c0e62 (127.15.208.3:35935)
I20260619 08:51:19.788400 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.787462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:40485 (local address 127.15.208.1:45415)
0619 08:51:19.787520 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.787522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.787528 (+ 6us) server_negotiation.cc:407] Connection header received
0619 08:51:19.787545 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.787546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.787550 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.787627 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.788152 (+ 525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.788153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.788154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.788158 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.788173 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.788178 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.788270 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.788311 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.788326 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:51:19.788405 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.787281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:45415 (local address 127.15.208.2:40485)
0619 08:51:19.787329 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.787341 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.787374 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.787648 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:19.787649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:19.787652 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:19.788036 (+ 384us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:19.788037 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:19.788043 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:19.788052 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:19.788257 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:19.788258 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:19.788259 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:51:19.788306 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 08:51:19.788318 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 08:51:19.788488 21287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.787359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:56551 (local address 127.15.208.3:35935)
0619 08:51:19.787398 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.787399 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.787596 (+ 197us) server_negotiation.cc:407] Connection header received
0619 08:51:19.787639 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.787640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.787643 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.787695 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.788291 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.788292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.788293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.788297 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.788310 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.788314 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.788366 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.788415 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.788426 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:51:19.788699 21298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.787283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:35935 (local address 127.15.208.2:56551)
0619 08:51:19.787529 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.787539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:19.787580 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:19.787695 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:19.787695 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:19.787712 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:19.788172 (+ 460us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:19.788172 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:19.788177 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:19.788185 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:19.788372 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:19.788372 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:19.788373 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:51:19.788416 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 08:51:19.788427 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":10,"thread_start_us":84,"threads_started":1}
I20260619 08:51:19.788759 20967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd4f8bede2b94eb3988510a576ec7edb" candidate_uuid: "d74db241c01b4b0f817bf6c2183d85e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88269a2cee184e7eb9c457c6b406bed8" is_pre_election: true
I20260619 08:51:19.788894 20967 raft_consensus.cc:2454] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74db241c01b4b0f817bf6c2183d85e1 in term 0.
I20260619 08:51:19.789012 21229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd4f8bede2b94eb3988510a576ec7edb" candidate_uuid: "d74db241c01b4b0f817bf6c2183d85e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c307afd096f7430286ed54e1152c0e62" is_pre_election: true
I20260619 08:51:19.789140 21229 raft_consensus.cc:2454] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74db241c01b4b0f817bf6c2183d85e1 in term 0.
I20260619 08:51:19.789350 21033 leader_election.cc:302] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c307afd096f7430286ed54e1152c0e62, d74db241c01b4b0f817bf6c2183d85e1; no voters:
I20260619 08:51:19.789489 21295 raft_consensus.cc:2781] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:19.789539 21295 raft_consensus.cc:482] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:19.789568 21295 raft_consensus.cc:3037] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:19.790144 21295 raft_consensus.cc:504] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 } } peers { permanent_uuid: "d74db241c01b4b0f817bf6c2183d85e1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46853 } } peers { permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 } }
I20260619 08:51:19.790275 21295 leader_election.cc:288] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [CANDIDATE]: Term 1 election: Requested vote from peers 88269a2cee184e7eb9c457c6b406bed8 (127.15.208.1:45415), c307afd096f7430286ed54e1152c0e62 (127.15.208.3:35935)
I20260619 08:51:19.790398 21229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd4f8bede2b94eb3988510a576ec7edb" candidate_uuid: "d74db241c01b4b0f817bf6c2183d85e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c307afd096f7430286ed54e1152c0e62"
I20260619 08:51:19.790419 20967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd4f8bede2b94eb3988510a576ec7edb" candidate_uuid: "d74db241c01b4b0f817bf6c2183d85e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88269a2cee184e7eb9c457c6b406bed8"
I20260619 08:51:19.790465 21229 raft_consensus.cc:3037] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:19.790479 20967 raft_consensus.cc:3037] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:19.791107 20967 raft_consensus.cc:2454] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74db241c01b4b0f817bf6c2183d85e1 in term 1.
I20260619 08:51:19.791114 21229 raft_consensus.cc:2454] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74db241c01b4b0f817bf6c2183d85e1 in term 1.
I20260619 08:51:19.791266 21031 leader_election.cc:302] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88269a2cee184e7eb9c457c6b406bed8, d74db241c01b4b0f817bf6c2183d85e1; no voters:
I20260619 08:51:19.791388 21295 raft_consensus.cc:2781] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:19.791507 21295 raft_consensus.cc:686] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 1 LEADER]: Becoming Leader. State: Replica: d74db241c01b4b0f817bf6c2183d85e1, State: Running, Role: LEADER
I20260619 08:51:19.791607 21295 consensus_queue.cc:237] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 } } peers { permanent_uuid: "d74db241c01b4b0f817bf6c2183d85e1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46853 } } peers { permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 } }
I20260619 08:51:19.792340 20824 catalog_manager.cc:5495] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d74db241c01b4b0f817bf6c2183d85e1 (127.15.208.2). New cstate: current_term: 1 leader_uuid: "d74db241c01b4b0f817bf6c2183d85e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74db241c01b4b0f817bf6c2183d85e1" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:19.800391 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.799549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42794 (local address 127.15.208.2:46853)
0619 08:51:19.799589 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.799591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.799607 (+ 16us) server_negotiation.cc:407] Connection header received
0619 08:51:19.799661 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.799662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.799671 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.799718 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.800254 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.800254 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.800254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.800257 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.800267 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.800270 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.800309 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.800338 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.800347 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:51:19.801910 20967 raft_consensus.cc:1261] T bd4f8bede2b94eb3988510a576ec7edb P 88269a2cee184e7eb9c457c6b406bed8 [term 1 FOLLOWER]: Refusing update from remote peer d74db241c01b4b0f817bf6c2183d85e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:19.801947 21229 raft_consensus.cc:1261] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62 [term 1 FOLLOWER]: Refusing update from remote peer d74db241c01b4b0f817bf6c2183d85e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:19.802225 21295 consensus_queue.cc:1035] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88269a2cee184e7eb9c457c6b406bed8" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:19.802337 21299 consensus_queue.cc:1035] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c307afd096f7430286ed54e1152c0e62" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:19.804579 21302 mvcc.cc:205] Tried to move back new op lower bound from 7298494790866452480 to 7298494790827073536. Current Snapshot: MvccSnapshot[applied={T|T < 7298494790866452480}]
I20260619 08:51:19.805222 21303 mvcc.cc:205] Tried to move back new op lower bound from 7298494790866452480 to 7298494790827073536. Current Snapshot: MvccSnapshot[applied={T|T < 7298494790866452480 or (T in {7298494790866452480})}]
I20260619 08:51:19.807157 20824 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:19.807272 20824 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:19.807979 20824 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4426725a526f42a29e40ac47ef3a75f4: Sending DeleteTablet for 3 replicas of tablet bd4f8bede2b94eb3988510a576ec7edb
I20260619 08:51:19.808343 21209 tablet_service.cc:1514] Processing DeleteTablet for tablet bd4f8bede2b94eb3988510a576ec7edb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:19 UTC) from {username='slave'} at 127.0.0.1:57904
I20260619 08:51:19.808477 20947 tablet_service.cc:1514] Processing DeleteTablet for tablet bd4f8bede2b94eb3988510a576ec7edb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:19 UTC) from {username='slave'} at 127.0.0.1:39314
I20260619 08:51:19.808588 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 20885
I20260619 08:51:19.810295 21078 tablet_service.cc:1514] Processing DeleteTablet for tablet bd4f8bede2b94eb3988510a576ec7edb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:19 UTC) from {username='slave'} at 127.0.0.1:42790
I20260619 08:51:19.810324 21311 tablet_replica.cc:330] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: stopping tablet replica
I20260619 08:51:19.810480 21311 raft_consensus.cc:2229] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:19.810607 21311 raft_consensus.cc:2258] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:19.810834 21313 tablet_replica.cc:330] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: stopping tablet replica
I20260619 08:51:19.810920 21313 raft_consensus.cc:2229] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:19.811040 21313 raft_consensus.cc:2258] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:19.811151 21311 ts_tablet_manager.cc:1905] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:19.811458 21313 ts_tablet_manager.cc:1905] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:19.812788 21311 ts_tablet_manager.cc:1918] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:19.812853 21311 log.cc:1196] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/bd4f8bede2b94eb3988510a576ec7edb
I20260619 08:51:19.813110 21311 ts_tablet_manager.cc:1939] T bd4f8bede2b94eb3988510a576ec7edb P c307afd096f7430286ed54e1152c0e62: Deleting consensus metadata
I20260619 08:51:19.813431 21313 ts_tablet_manager.cc:1918] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:19.813508 21313 log.cc:1196] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/bd4f8bede2b94eb3988510a576ec7edb
I20260619 08:51:19.813630 20811 catalog_manager.cc:4841] TS c307afd096f7430286ed54e1152c0e62 (127.15.208.3:35935): tablet bd4f8bede2b94eb3988510a576ec7edb (table test-table [id=7f9707a886dc4f3798144ae4ec9a12fa]) successfully deleted
I20260619 08:51:19.813778 21313 ts_tablet_manager.cc:1939] T bd4f8bede2b94eb3988510a576ec7edb P d74db241c01b4b0f817bf6c2183d85e1: Deleting consensus metadata
I20260619 08:51:19.814157 20809 catalog_manager.cc:4841] TS d74db241c01b4b0f817bf6c2183d85e1 (127.15.208.2:46853): tablet bd4f8bede2b94eb3988510a576ec7edb (table test-table [id=7f9707a886dc4f3798144ae4ec9a12fa]) successfully deleted
W20260619 08:51:19.814702 20809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:45415 (error 108)
I20260619 08:51:19.815243 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.815102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:45415 (local address 127.0.0.1:39318)
0619 08:51:19.815166 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:19.815209 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 08:51:19.815299 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 21016
W20260619 08:51:19.817588 20809 catalog_manager.cc:4584] TS 88269a2cee184e7eb9c457c6b406bed8 (127.15.208.1:45415): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd4f8bede2b94eb3988510a576ec7edb: Network error: Client connection negotiation failed: client connection to 127.15.208.1:45415: connect: Connection refused (error 111)
I20260619 08:51:19.820048 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 21147
I20260619 08:51:19.824432 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 20794
2026-06-19T08:51:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (592 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 08:51:19.844583 16192 mini_oidc.cc:152] Starting JWKS server
I20260619 08:51:19.845676 16192 webserver.cc:460] Webserver started at https://127.0.0.1:41187/ using document root <none> and password file <none>
I20260619 08:51:19.845785 16192 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 08:51:19.845976 16192 webserver.cc:460] Webserver started at http://127.0.0.1:34169/ using document root <none> and password file <none>
2026-06-19T08:51:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:19Z Disabled control of system clock
I20260619 08:51:19.853503 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:43861
--webserver_interface=127.15.208.62
--webserver_port=0
--builtin_ntp_servers=127.15.208.20:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:43861
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41187/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/testchainca.cert with env {}
W20260619 08:51:19.932823 21320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:19.933036 21320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:19.933059 21320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:19.934136 21320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:19.934170 21320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:19.934183 21320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:19.934195 21320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:19.934207 21320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:51:19.934219 21320 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:51:19.934231 21320 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41187/jwks.json
I20260619 08:51:19.935712 21320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:43861
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:43861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41187/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:19.935904 21320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:19.936070 21320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:19.940845 21325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:19.940845 21328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:19.940889 21326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:19.941123 21320 server_base.cc:1034] running on GCE node
I20260619 08:51:19.941360 21320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:19.941648 21320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:19.942791 21320 hybrid_clock.cc:648] HybridClock initialized: now 1781859079942776 us; error 32 us; skew 500 ppm
I20260619 08:51:19.943953 21320 webserver.cc:460] Webserver started at http://127.15.208.62:38439/ using document root <none> and password file <none>
I20260619 08:51:19.944154 21320 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:19.944197 21320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:19.944285 21320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:19.945330 21320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "0f526a819dac4075850dc0403209fbf4"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "b82931b84a5f9f1b64117b7df10f82b7"
server_key_iv: "c22a7e0601fe525a793a67a553e24459"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.945652 21320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "0f526a819dac4075850dc0403209fbf4"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "b82931b84a5f9f1b64117b7df10f82b7"
server_key_iv: "c22a7e0601fe525a793a67a553e24459"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.946928 21320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:19.947750 21334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:19.947953 21320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:19.948026 21320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "0f526a819dac4075850dc0403209fbf4"
format_stamp: "Formatted at 2026-06-19 08:51:19 on dist-test-slave-p0h9"
server_key: "b82931b84a5f9f1b64117b7df10f82b7"
server_key_iv: "c22a7e0601fe525a793a67a553e24459"
server_key_version: "encryptionkey@0"
I20260619 08:51:19.948122 21320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:19.960717 21320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:19.961081 21320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:19.961226 21320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:19.965008 21320 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:43861
I20260619 08:51:19.965042 21386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:43861 every 8 connection(s)
I20260619 08:51:19.965370 21320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:19.966305 21387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:19.967561 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 21320
I20260619 08:51:19.967675 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:19.967880 16192 external_mini_cluster.cc:1428] Setting key 92031b926075b5314e3b5157db25a89d
I20260619 08:51:19.969128 21387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4: Bootstrap starting.
I20260619 08:51:19.969702 21387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:19.970001 21387 log.cc:824] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:19.970669 21387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4: No bootstrap required, opened a new log
I20260619 08:51:19.972352 21387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f526a819dac4075850dc0403209fbf4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43861 } }
I20260619 08:51:19.972503 21387 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:19.972553 21387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f526a819dac4075850dc0403209fbf4, State: Initialized, Role: FOLLOWER
I20260619 08:51:19.972689 21387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f526a819dac4075850dc0403209fbf4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43861 } }
I20260619 08:51:19.972815 21387 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:19.972855 21387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:19.972903 21387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:19.973381 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:19.968379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60974 (local address 127.15.208.62:43861)
0619 08:51:19.968593 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:19.968600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:19.968632 (+ 32us) server_negotiation.cc:407] Connection header received
0619 08:51:19.969103 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:19.969110 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:19.969193 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:19.969363 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:19.969543 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.970509 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.971359 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:19.971786 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:19.971924 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:19.972638 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:19.972642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:19.972642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:19.972871 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:19.972918 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:19.972932 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:19.973020 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:19.973095 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 08:51:19.973178 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260619 08:51:19.973582 21387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f526a819dac4075850dc0403209fbf4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43861 } }
I20260619 08:51:19.973687 21387 leader_election.cc:302] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f526a819dac4075850dc0403209fbf4; no voters:
I20260619 08:51:19.973856 21387 leader_election.cc:288] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:19.973992 21392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:19.974107 21392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [term 1 LEADER]: Becoming Leader. State: Replica: 0f526a819dac4075850dc0403209fbf4, State: Running, Role: LEADER
I20260619 08:51:19.974135 21387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:19.974185 21392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f526a819dac4075850dc0403209fbf4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43861 } }
I20260619 08:51:19.974252 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:51:19.974525 21392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f526a819dac4075850dc0403209fbf4. Latest consensus state: current_term: 1 leader_uuid: "0f526a819dac4075850dc0403209fbf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f526a819dac4075850dc0403209fbf4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43861 } } }
I20260619 08:51:19.974535 21393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f526a819dac4075850dc0403209fbf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f526a819dac4075850dc0403209fbf4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43861 } } }
I20260619 08:51:19.974619 21392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:19.974632 21393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:19.975309 21398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:19.975821 21398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:19.977288 21398 catalog_manager.cc:1347] Generated new cluster ID: 7cd1fb62733744e2adab16bc44962c5d
I20260619 08:51:19.977344 21398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:19.995406 21398 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:19.995957 21398 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:20.005409 21398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f526a819dac4075850dc0403209fbf4: Generated new TSK 0
I20260619 08:51:20.005635 21398 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 08:51:20.041260 21417 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 08:51:20.041366 21417 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 08:51:20.041512 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:20.018478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.15.208.62:43861)
0619 08:51:20.018550 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:20.018552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:20.018708 (+ 156us) server_negotiation.cc:407] Connection header received
0619 08:51:20.018814 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:20.018815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:20.018825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:20.018886 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:51:20.019066 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.019536 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.020673 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.020927 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.020979 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.021244 (+ 265us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:51:20.041249 (+ 20005us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 08:51:20.041428 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60990: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260619 08:51:20.041582 21390 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60990: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 08:51:20.041877 21416 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:20.018421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43861 (local address 127.0.0.1:60990)
0619 08:51:20.018627 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:20.018639 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:20.018718 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:20.018914 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:20.018915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:20.018917 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:51:20.018981 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:20.018983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.019629 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.019630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:20.020588 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:20.020591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.021014 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.021015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:20.021075 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.021098 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:51:20.041398 (+ 20300us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 08:51:20.041520 (+ 122us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.208.62:43861: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.208.62:43861: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 08:51:20.042299 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 21320
2026-06-19T08:51:20Z chronyd exiting
I20260619 08:51:20.078251 16192 test_util.cc:175] -----------------------------------------------
I20260619 08:51:20.078328 16192 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781859057751076-16192-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (234 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 08:51:20.079073 16192 mini_oidc.cc:152] Starting JWKS server
I20260619 08:51:20.079764 16192 webserver.cc:460] Webserver started at https://127.0.0.1:45333/ using document root <none> and password file <none>
I20260619 08:51:20.079804 16192 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 08:51:20.079989 16192 webserver.cc:460] Webserver started at http://127.0.0.1:43717/ using document root <none> and password file <none>
2026-06-19T08:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:20Z Disabled control of system clock
I20260619 08:51:20.087500 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:41069
--webserver_interface=127.15.208.62
--webserver_port=0
--builtin_ntp_servers=127.15.208.20:34967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:41069
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45333/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/testchainca.cert with env {}
W20260619 08:51:20.169189 21423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:20.169405 21423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:20.169437 21423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:20.170624 21423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:20.170670 21423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:20.170684 21423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:20.170699 21423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:20.170712 21423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:51:20.170727 21423 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:51:20.170743 21423 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45333/jwks.json
I20260619 08:51:20.172305 21423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:34967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:41069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:41069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45333/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:20.172516 21423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:20.172681 21423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:20.177654 21428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:20.177719 21431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:20.177772 21423 server_base.cc:1034] running on GCE node
W20260619 08:51:20.177719 21429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:20.178149 21423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:20.178385 21423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:20.179538 21423 hybrid_clock.cc:648] HybridClock initialized: now 1781859080179511 us; error 47 us; skew 500 ppm
I20260619 08:51:20.180979 21423 webserver.cc:460] Webserver started at http://127.15.208.62:33111/ using document root <none> and password file <none>
I20260619 08:51:20.181237 21423 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:20.181303 21423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:20.181454 21423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:20.182623 21423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "8277ca5b25da4136be9ca17de5488575"
format_stamp: "Formatted at 2026-06-19 08:51:20 on dist-test-slave-p0h9"
server_key: "8e5875e5f7e7024ad0620f8242c3fa11"
server_key_iv: "67c956f94c52d8619694d3bd1cb4b6f6"
server_key_version: "encryptionkey@0"
I20260619 08:51:20.183005 21423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "8277ca5b25da4136be9ca17de5488575"
format_stamp: "Formatted at 2026-06-19 08:51:20 on dist-test-slave-p0h9"
server_key: "8e5875e5f7e7024ad0620f8242c3fa11"
server_key_iv: "67c956f94c52d8619694d3bd1cb4b6f6"
server_key_version: "encryptionkey@0"
I20260619 08:51:20.184302 21423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:20.185148 21437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:20.185309 21423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:20.185415 21423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "8277ca5b25da4136be9ca17de5488575"
format_stamp: "Formatted at 2026-06-19 08:51:20 on dist-test-slave-p0h9"
server_key: "8e5875e5f7e7024ad0620f8242c3fa11"
server_key_iv: "67c956f94c52d8619694d3bd1cb4b6f6"
server_key_version: "encryptionkey@0"
I20260619 08:51:20.185487 21423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:20.199692 21423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:20.200013 21423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:20.200176 21423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:20.203956 21423 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:41069
I20260619 08:51:20.204026 21489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:41069 every 8 connection(s)
I20260619 08:51:20.204360 21423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:20.205076 21490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:20.207736 21490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575: Bootstrap starting.
I20260619 08:51:20.208323 21490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:20.208624 21490 log.cc:824] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:20.209314 21490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575: No bootstrap required, opened a new log
I20260619 08:51:20.210930 21490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8277ca5b25da4136be9ca17de5488575" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41069 } }
I20260619 08:51:20.211066 21490 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:20.211088 21490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8277ca5b25da4136be9ca17de5488575, State: Initialized, Role: FOLLOWER
I20260619 08:51:20.211261 21490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8277ca5b25da4136be9ca17de5488575" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41069 } }
I20260619 08:51:20.211319 21490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:20.211339 21490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:20.211405 21490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:20.211871 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 21423
I20260619 08:51:20.211978 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:20.212123 21490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8277ca5b25da4136be9ca17de5488575" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41069 } }
I20260619 08:51:20.212186 16192 external_mini_cluster.cc:1428] Setting key a4725fcfddcd2860fa4825a868e9d03b
I20260619 08:51:20.212232 21490 leader_election.cc:302] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8277ca5b25da4136be9ca17de5488575; no voters:
I20260619 08:51:20.212395 21490 leader_election.cc:288] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:20.212420 21493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:20.212528 21493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [term 1 LEADER]: Becoming Leader. State: Replica: 8277ca5b25da4136be9ca17de5488575, State: Running, Role: LEADER
I20260619 08:51:20.212612 21493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8277ca5b25da4136be9ca17de5488575" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41069 } }
I20260619 08:51:20.212911 21490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:20.212954 21494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8277ca5b25da4136be9ca17de5488575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8277ca5b25da4136be9ca17de5488575" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41069 } } }
I20260619 08:51:20.213021 21494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:20.213416 21499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:20.214020 21499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:20.212929 21493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8277ca5b25da4136be9ca17de5488575. Latest consensus state: current_term: 1 leader_uuid: "8277ca5b25da4136be9ca17de5488575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8277ca5b25da4136be9ca17de5488575" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 41069 } } }
I20260619 08:51:20.214450 21493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:20.215771 21499 catalog_manager.cc:1347] Generated new cluster ID: 10e73e2535b244c09cf7877153d3dfbd
I20260619 08:51:20.215823 21499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:20.218520 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:20.212714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36112 (local address 127.15.208.62:41069)
0619 08:51:20.213023 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:20.213032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:20.213065 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:51:20.213186 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:20.213193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:20.213272 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:20.213673 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:20.213772 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.215240 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.216323 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.216959 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.217071 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.217802 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:20.217806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:20.217807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:20.218004 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:20.218043 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:20.218057 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:20.218116 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:20.218216 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 08:51:20.218293 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":72,"threads_started":1}
I20260619 08:51:20.219481 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:51:20.223814 21499 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:20.224488 21499 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:20.246979 21499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8277ca5b25da4136be9ca17de5488575: Generated new TSK 0
I20260619 08:51:20.247197 21499 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 08:51:20.272686 21520 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 08:51:20.272810 21520 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 08:51:20.273231 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:20.258341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36128 (local address 127.15.208.62:41069)
0619 08:51:20.258413 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:20.258416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:20.258519 (+ 103us) server_negotiation.cc:407] Connection header received
0619 08:51:20.258586 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:20.258587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:20.258597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:20.258659 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:51:20.258814 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.259359 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.260516 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.260781 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.260856 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.261102 (+ 246us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:51:20.273035 (+ 11933us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45333/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 08:51:20.273177 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36128: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45333/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 08:51:20.273309 21497 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36128: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45333/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 08:51:20.273409 21519 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:20.258273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:41069 (local address 127.0.0.1:36128)
0619 08:51:20.258439 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:20.258450 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:20.258521 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:20.258677 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:20.258678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:20.258679 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:51:20.258747 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:20.258749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.259434 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.259435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:20.260408 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:20.260411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.260870 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.260871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:20.260935 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.260942 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:51:20.273160 (+ 12218us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45333/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 08:51:20.273303 (+ 143us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.208.62:41069: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45333/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":112,"threads_started":1}
I20260619 08:51:20.273715 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 21423
2026-06-19T08:51:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (232 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 08:51:20.311306 16192 mini_oidc.cc:152] Starting JWKS server
I20260619 08:51:20.311946 16192 webserver.cc:460] Webserver started at https://127.0.0.1:36167/ using document root <none> and password file <none>
I20260619 08:51:20.311986 16192 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 08:51:20.312160 16192 webserver.cc:460] Webserver started at http://127.0.0.1:45439/ using document root <none> and password file <none>
2026-06-19T08:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:20Z Disabled control of system clock
I20260619 08:51:20.319824 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:34901
--webserver_interface=127.15.208.62
--webserver_port=0
--builtin_ntp_servers=127.15.208.20:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:34901
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36167/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 08:51:20.401978 21526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:20.402213 21526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:20.402249 21526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:20.403465 21526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:20.403524 21526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:20.403548 21526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:20.403568 21526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:20.403616 21526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:51:20.403647 21526 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:51:20.403673 21526 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36167/jwks.json
I20260619 08:51:20.405133 21526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:34901
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:34901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36167/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:20.405402 21526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:20.405689 21526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:20.410362 21531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:20.410389 21532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:20.410398 21534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:20.410506 21526 server_base.cc:1034] running on GCE node
I20260619 08:51:20.410748 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:20.411010 21526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:20.412158 21526 hybrid_clock.cc:648] HybridClock initialized: now 1781859080412143 us; error 34 us; skew 500 ppm
I20260619 08:51:20.413503 21526 webserver.cc:460] Webserver started at http://127.15.208.62:45247/ using document root <none> and password file <none>
I20260619 08:51:20.413766 21526 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:20.413815 21526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:20.413960 21526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:20.415030 21526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "7073d4cdefe9432e9ef4ccb119761769"
format_stamp: "Formatted at 2026-06-19 08:51:20 on dist-test-slave-p0h9"
server_key: "ed035bbb86590ebaacf0c1f570a83729"
server_key_iv: "6a41f5bd97bbc13b57d2c3afb401e7f8"
server_key_version: "encryptionkey@0"
I20260619 08:51:20.415409 21526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "7073d4cdefe9432e9ef4ccb119761769"
format_stamp: "Formatted at 2026-06-19 08:51:20 on dist-test-slave-p0h9"
server_key: "ed035bbb86590ebaacf0c1f570a83729"
server_key_iv: "6a41f5bd97bbc13b57d2c3afb401e7f8"
server_key_version: "encryptionkey@0"
I20260619 08:51:20.416678 21526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:20.417431 21540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:20.417606 21526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:20.417676 21526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "7073d4cdefe9432e9ef4ccb119761769"
format_stamp: "Formatted at 2026-06-19 08:51:20 on dist-test-slave-p0h9"
server_key: "ed035bbb86590ebaacf0c1f570a83729"
server_key_iv: "6a41f5bd97bbc13b57d2c3afb401e7f8"
server_key_version: "encryptionkey@0"
I20260619 08:51:20.417759 21526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:20.438457 21526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:20.438791 21526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:20.438941 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:20.442986 21526 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:34901
I20260619 08:51:20.443027 21592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:34901 every 8 connection(s)
I20260619 08:51:20.443406 21526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:20.443890 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 21526
I20260619 08:51:20.443997 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:20.444190 16192 external_mini_cluster.cc:1428] Setting key c7297191ac73249086daebdf5a821d03
I20260619 08:51:20.444219 21593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:20.447710 21593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769: Bootstrap starting.
I20260619 08:51:20.448338 21593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:20.448628 21593 log.cc:824] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:20.449358 21593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769: No bootstrap required, opened a new log
I20260619 08:51:20.450513 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:20.445047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56118 (local address 127.15.208.62:34901)
0619 08:51:20.445237 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:20.445246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:20.445281 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:51:20.445842 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:20.445852 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:20.445937 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:20.446092 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:20.446312 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.447662 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.448554 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.449031 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.449178 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.449903 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:20.449907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:20.449907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:20.450082 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:20.450115 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:20.450125 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:20.450194 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:20.450268 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 08:51:20.450346 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":59,"threads_started":1}
I20260619 08:51:20.451510 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:51:20.451575 21593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7073d4cdefe9432e9ef4ccb119761769" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34901 } }
I20260619 08:51:20.451756 21593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:20.451802 21593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7073d4cdefe9432e9ef4ccb119761769, State: Initialized, Role: FOLLOWER
I20260619 08:51:20.451921 21593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7073d4cdefe9432e9ef4ccb119761769" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34901 } }
I20260619 08:51:20.452013 21593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:20.452056 21593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:20.452105 21593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:20.452775 21593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7073d4cdefe9432e9ef4ccb119761769" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34901 } }
I20260619 08:51:20.452900 21593 leader_election.cc:302] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7073d4cdefe9432e9ef4ccb119761769; no voters:
I20260619 08:51:20.453118 21593 leader_election.cc:288] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:20.453150 21598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:20.453294 21598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [term 1 LEADER]: Becoming Leader. State: Replica: 7073d4cdefe9432e9ef4ccb119761769, State: Running, Role: LEADER
I20260619 08:51:20.453373 21593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:20.453378 21598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7073d4cdefe9432e9ef4ccb119761769" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34901 } }
I20260619 08:51:20.453783 21600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7073d4cdefe9432e9ef4ccb119761769. Latest consensus state: current_term: 1 leader_uuid: "7073d4cdefe9432e9ef4ccb119761769" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7073d4cdefe9432e9ef4ccb119761769" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34901 } } }
I20260619 08:51:20.453845 21599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7073d4cdefe9432e9ef4ccb119761769" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7073d4cdefe9432e9ef4ccb119761769" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34901 } } }
I20260619 08:51:20.453956 21599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:20.454156 21600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:20.454278 21605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:20.454804 21605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:20.456305 21605 catalog_manager.cc:1347] Generated new cluster ID: 70e6ff614531438790e698c83ec56f7a
I20260619 08:51:20.456357 21605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:20.473848 21605 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:20.474372 21605 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:20.478420 21605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7073d4cdefe9432e9ef4ccb119761769: Generated new TSK 0
I20260619 08:51:20.478647 21605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:51:20.491216 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:20.487218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.15.208.62:34901)
0619 08:51:20.487283 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:20.487286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:20.487397 (+ 111us) server_negotiation.cc:407] Connection header received
0619 08:51:20.487523 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:20.487524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:20.487536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:20.487616 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:51:20.487763 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.488271 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.489157 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.489365 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.489475 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.489703 (+ 228us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:51:20.490984 (+ 1281us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36167/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 08:51:20.491137 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56120: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36167/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":34}
W20260619 08:51:20.491282 21596 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56120: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36167/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 08:51:20.491391 21622 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:20.487144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:34901 (local address 127.0.0.1:56120)
0619 08:51:20.487316 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:20.487327 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:20.487417 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:20.487616 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:20.487617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:20.487618 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:51:20.487679 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:20.487681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.488363 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.488364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:20.489072 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:20.489074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.489506 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.489508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:20.489580 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.489585 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:51:20.491089 (+ 1504us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36167/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 08:51:20.491177 (+ 88us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.208.62:34901: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36167/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260619 08:51:20.491729 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 21526
E20260619 08:51:20.502532 21623 webserver.cc:533] Webserver: sslize failed:
E20260619 08:51:20.502600 21623 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T08:51:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (217 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T08:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:20Z Disabled control of system clock
I20260619 08:51:20.536427 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:33379
--webserver_interface=127.15.208.62
--webserver_port=0
--builtin_ntp_servers=127.15.208.20:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:33379
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/testchainca.cert with env {}
W20260619 08:51:20.616227 21627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:20.616425 21627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:20.616446 21627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:20.617547 21627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:20.617580 21627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:20.617594 21627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:20.617606 21627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:20.617619 21627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:51:20.617630 21627 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:51:20.617642 21627 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 08:51:20.619086 21627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:33379
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:33379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:20.619274 21627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:20.619429 21627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:20.624132 21633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:20.624159 21635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:20.624182 21632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:20.624522 21627 server_base.cc:1034] running on GCE node
I20260619 08:51:20.624815 21627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:20.625108 21627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:20.626253 21627 hybrid_clock.cc:648] HybridClock initialized: now 1781859080626235 us; error 35 us; skew 500 ppm
I20260619 08:51:20.627542 21627 webserver.cc:460] Webserver started at http://127.15.208.62:44013/ using document root <none> and password file <none>
I20260619 08:51:20.627789 21627 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:20.627835 21627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:20.627983 21627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:20.629070 21627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "5915f549840b4c7491a4e0e4862faf8c"
format_stamp: "Formatted at 2026-06-19 08:51:20 on dist-test-slave-p0h9"
server_key: "0f8c0d1376373710e79d8bdc49ed1025"
server_key_iv: "c0512599bf74509cc585e7a901ca1f80"
server_key_version: "encryptionkey@0"
I20260619 08:51:20.629412 21627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "5915f549840b4c7491a4e0e4862faf8c"
format_stamp: "Formatted at 2026-06-19 08:51:20 on dist-test-slave-p0h9"
server_key: "0f8c0d1376373710e79d8bdc49ed1025"
server_key_iv: "c0512599bf74509cc585e7a901ca1f80"
server_key_version: "encryptionkey@0"
I20260619 08:51:20.630698 21627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:20.631379 21641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:20.631547 21627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:20.631651 21627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "5915f549840b4c7491a4e0e4862faf8c"
format_stamp: "Formatted at 2026-06-19 08:51:20 on dist-test-slave-p0h9"
server_key: "0f8c0d1376373710e79d8bdc49ed1025"
server_key_iv: "c0512599bf74509cc585e7a901ca1f80"
server_key_version: "encryptionkey@0"
I20260619 08:51:20.631726 21627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:20.647738 21627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:20.648065 21627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:20.648226 21627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:20.652304 21627 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:33379
I20260619 08:51:20.652362 21693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:33379 every 8 connection(s)
I20260619 08:51:20.652701 21627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:20.653646 21694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:20.655854 21694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c: Bootstrap starting.
I20260619 08:51:20.656437 21694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:20.656798 21694 log.cc:824] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:20.657444 21694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c: No bootstrap required, opened a new log
I20260619 08:51:20.659013 21694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5915f549840b4c7491a4e0e4862faf8c" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33379 } }
I20260619 08:51:20.659154 21694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:20.659202 21694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5915f549840b4c7491a4e0e4862faf8c, State: Initialized, Role: FOLLOWER
I20260619 08:51:20.659353 21694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5915f549840b4c7491a4e0e4862faf8c" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33379 } }
I20260619 08:51:20.659438 21694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:20.659480 21694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:20.659530 21694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:20.660044 21694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5915f549840b4c7491a4e0e4862faf8c" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33379 } }
I20260619 08:51:20.660161 21694 leader_election.cc:302] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5915f549840b4c7491a4e0e4862faf8c; no voters:
I20260619 08:51:20.660339 21694 leader_election.cc:288] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:20.660382 21697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:20.660563 21697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [term 1 LEADER]: Becoming Leader. State: Replica: 5915f549840b4c7491a4e0e4862faf8c, State: Running, Role: LEADER
I20260619 08:51:20.660634 21694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:20.660684 21697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5915f549840b4c7491a4e0e4862faf8c" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33379 } }
I20260619 08:51:20.660799 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 21627
I20260619 08:51:20.660903 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:20.661054 21697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5915f549840b4c7491a4e0e4862faf8c. Latest consensus state: current_term: 1 leader_uuid: "5915f549840b4c7491a4e0e4862faf8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5915f549840b4c7491a4e0e4862faf8c" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33379 } } }
I20260619 08:51:20.661094 16192 external_mini_cluster.cc:1428] Setting key 25a627395c1d1d3acdb7a1f663c73a0f
I20260619 08:51:20.661127 21697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:20.661266 21698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5915f549840b4c7491a4e0e4862faf8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5915f549840b4c7491a4e0e4862faf8c" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33379 } } }
I20260619 08:51:20.661358 21698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:20.661866 21704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:20.662334 21704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:20.663906 21704 catalog_manager.cc:1347] Generated new cluster ID: a90dd7f471444436aec12c90177b2f26
I20260619 08:51:20.663955 21704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:20.667520 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:20.661836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55362 (local address 127.15.208.62:33379)
0619 08:51:20.662235 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:20.662244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:20.662275 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:51:20.662388 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:20.662395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:20.662466 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:20.662723 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:20.662953 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.664298 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.665413 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.666003 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.666116 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.666806 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:20.666809 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:20.666810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:20.667030 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:20.667070 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:20.667084 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:20.667147 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:20.667244 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 08:51:20.667318 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":91,"threads_started":1}
I20260619 08:51:20.668453 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:51:20.674180 21704 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:20.674686 21704 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:20.678474 21704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5915f549840b4c7491a4e0e4862faf8c: Generated new TSK 0
I20260619 08:51:20.678658 21704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:51:20.739820 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:20.684389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55378 (local address 127.15.208.62:33379)
0619 08:51:20.684430 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:20.684433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:20.684552 (+ 119us) server_negotiation.cc:407] Connection header received
0619 08:51:20.684636 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:20.684637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:20.684665 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:20.684772 (+ 107us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:51:20.684910 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.685391 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.686191 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.686400 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.686472 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.686691 (+ 219us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:51:20.739523 (+ 52832us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 08:51:20.739720 (+ 197us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55378: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:51:20.739902 21708 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55378: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 08:51:20.739908 21723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:20.684340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:33379 (local address 127.0.0.1:55378)
0619 08:51:20.684484 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:20.684493 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:20.684572 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:20.684771 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:20.684772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:20.684774 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:51:20.684831 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:20.684832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.685488 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.685489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:20.686104 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:20.686106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:20.686490 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:20.686492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:20.686547 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:20.686557 (+ 10us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:51:20.739665 (+ 53108us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 08:51:20.739790 (+ 125us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.208.62:33379: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260619 08:51:20.740319 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 21627
2026-06-19T08:51:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (227 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (85 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:20 dist-test-slave-p0h9 krb5kdc[21735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:20 dist-test-slave-p0h9 krb5kdc[21735](info): set up 2 sockets
Jun 19 08:51:20 dist-test-slave-p0h9 krb5kdc[21735](info): commencing operation
krb5kdc: starting...
W20260619 08:51:22.005692 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260619 08:51:22.258140 16192 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.252s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260619 08:51:22.556653 16192 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.298s user 0.001s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260619 08:51:22.873919 16192 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.317s user 0.002s sys 0.000s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/test-user.keytab.
W20260619 08:51:23.772945 16192 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.899s user 0.002s sys 0.000s
Jun 19 08:51:23 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:23.815534 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:33283
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:33283
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:23.898981 21748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:23.899214 21748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:23.899237 21748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:23.900524 21748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:23.900573 21748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:23.900589 21748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:23.900638 21748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:23.900660 21748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:23.902266 21748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:33283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:33283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:23.902530 21748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:23.902772 21748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:23.907617 21756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:23.907645 21754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:23.907649 21753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:23.907717 21748 server_base.cc:1034] running on GCE node
I20260619 08:51:23.908069 21748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:23.908313 21748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:23.909469 21748 hybrid_clock.cc:648] HybridClock initialized: now 1781859083909451 us; error 35 us; skew 500 ppm
Jun 19 08:51:23 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:23.919207 21748 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:23.919629 21748 webserver.cc:460] Webserver started at http://127.15.208.62:39561/ using document root <none> and password file <none>
I20260619 08:51:23.919872 21748 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:23.919920 21748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:23.920100 21748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:23.921442 21748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "5a6578d8f81e49e1ad6ba707c0b616ee"
format_stamp: "Formatted at 2026-06-19 08:51:23 on dist-test-slave-p0h9"
server_key: "8efe612490ef0728bb332a6658dc274b"
server_key_iv: "f0067a730d5d006dda1c99f3530c0a40"
server_key_version: "encryptionkey@0"
I20260619 08:51:23.921815 21748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "5a6578d8f81e49e1ad6ba707c0b616ee"
format_stamp: "Formatted at 2026-06-19 08:51:23 on dist-test-slave-p0h9"
server_key: "8efe612490ef0728bb332a6658dc274b"
server_key_iv: "f0067a730d5d006dda1c99f3530c0a40"
server_key_version: "encryptionkey@0"
I20260619 08:51:23.923197 21748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:23.924041 21763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:23.924252 21748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:23.924363 21748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "5a6578d8f81e49e1ad6ba707c0b616ee"
format_stamp: "Formatted at 2026-06-19 08:51:23 on dist-test-slave-p0h9"
server_key: "8efe612490ef0728bb332a6658dc274b"
server_key_iv: "f0067a730d5d006dda1c99f3530c0a40"
server_key_version: "encryptionkey@0"
I20260619 08:51:23.924439 21748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:23.938683 21748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:23.940583 21748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:23.940795 21748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:23.945153 21748 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:33283
I20260619 08:51:23.945206 21815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:33283 every 8 connection(s)
I20260619 08:51:23.945569 21748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:23.946431 21816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:23.948639 21816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee: Bootstrap starting.
I20260619 08:51:23.949280 21816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:23.949527 21816 log.cc:824] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:23.950176 21816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee: No bootstrap required, opened a new log
I20260619 08:51:23.950630 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 21748
I20260619 08:51:23.950722 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:23.950889 16192 external_mini_cluster.cc:1428] Setting key a4d44b0ebac52d029119004c72f60d61
I20260619 08:51:23.952131 21816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6578d8f81e49e1ad6ba707c0b616ee" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33283 } }
I20260619 08:51:23.952302 21816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:23.952392 21816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6578d8f81e49e1ad6ba707c0b616ee, State: Initialized, Role: FOLLOWER
I20260619 08:51:23.952569 21816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6578d8f81e49e1ad6ba707c0b616ee" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33283 } }
I20260619 08:51:23.952659 21816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:23.952693 21816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:23.952780 21816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:23.953702 21816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6578d8f81e49e1ad6ba707c0b616ee" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33283 } }
I20260619 08:51:23.953785 21816 leader_election.cc:302] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a6578d8f81e49e1ad6ba707c0b616ee; no voters:
I20260619 08:51:23.953970 21816 leader_election.cc:288] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:23.954018 21821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:23.954186 21821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [term 1 LEADER]: Becoming Leader. State: Replica: 5a6578d8f81e49e1ad6ba707c0b616ee, State: Running, Role: LEADER
I20260619 08:51:23.954231 21816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:23.954309 21821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6578d8f81e49e1ad6ba707c0b616ee" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33283 } }
I20260619 08:51:23.954691 21821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a6578d8f81e49e1ad6ba707c0b616ee. Latest consensus state: current_term: 1 leader_uuid: "5a6578d8f81e49e1ad6ba707c0b616ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6578d8f81e49e1ad6ba707c0b616ee" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33283 } } }
I20260619 08:51:23.954896 21821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:23.954841 21822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a6578d8f81e49e1ad6ba707c0b616ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6578d8f81e49e1ad6ba707c0b616ee" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 33283 } } }
I20260619 08:51:23.955108 21822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:23.955487 21830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:23.955997 21830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:23.957585 21830 catalog_manager.cc:1347] Generated new cluster ID: 7428206bf4104dccbc26ef463148806e
I20260619 08:51:23.957635 21830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:51:23 dist-test-slave-p0h9 krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:23.962889 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:23.951403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46046 (local address 127.15.208.62:33283)
0619 08:51:23.951576 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:23.951583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:23.951606 (+ 23us) server_negotiation.cc:407] Connection header received
0619 08:51:23.952112 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:23.952117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:23.952162 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:23.952272 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:23.953110 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:23.954182 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:23.955289 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:23.956086 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:23.956192 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:23.959310 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:23.959314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:23.959314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:23.959326 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:23.961273 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:23.961683 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:23.961686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:23.961687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:23.961799 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:23.962006 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:23.962007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:23.962007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:23.962287 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:23.962430 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:23.962576 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 08:51:23.962700 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260619 08:51:23.987949 21830 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:23.988756 21830 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:23.993705 21830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a6578d8f81e49e1ad6ba707c0b616ee: Generated new TSK 0
I20260619 08:51:23.993898 21830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:51:24.019905 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33283
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:24.100169 21842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:24.100404 21842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:24.100441 21842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:24.101625 21842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:24.101697 21842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:24.101745 21842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:24.103591 21842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:24.103896 21842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:24.104162 21842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:24.109129 21847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:24.109126 21850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:24.109198 21848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:24.109253 21842 server_base.cc:1034] running on GCE node
I20260619 08:51:24.109609 21842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:24.109807 21842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:24.110949 21842 hybrid_clock.cc:648] HybridClock initialized: now 1781859084110919 us; error 46 us; skew 500 ppm
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:24.113523 21842 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:51:24.113970 21842 webserver.cc:460] Webserver started at http://127.15.208.1:34243/ using document root <none> and password file <none>
I20260619 08:51:24.114147 21842 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:24.114187 21842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:24.114307 21842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:24.115312 21842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "82524820224c4a61a11c352bc276c3b3"
format_stamp: "Formatted at 2026-06-19 08:51:24 on dist-test-slave-p0h9"
server_key: "27017b34c69c74ae819275a31891bd8f"
server_key_iv: "60196166811fffcadf46862a77a23659"
server_key_version: "encryptionkey@0"
I20260619 08:51:24.115630 21842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "82524820224c4a61a11c352bc276c3b3"
format_stamp: "Formatted at 2026-06-19 08:51:24 on dist-test-slave-p0h9"
server_key: "27017b34c69c74ae819275a31891bd8f"
server_key_iv: "60196166811fffcadf46862a77a23659"
server_key_version: "encryptionkey@0"
I20260619 08:51:24.117079 21842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:51:24.117787 21857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:24.117974 21842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:24.118058 21842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "82524820224c4a61a11c352bc276c3b3"
format_stamp: "Formatted at 2026-06-19 08:51:24 on dist-test-slave-p0h9"
server_key: "27017b34c69c74ae819275a31891bd8f"
server_key_iv: "60196166811fffcadf46862a77a23659"
server_key_version: "encryptionkey@0"
I20260619 08:51:24.118111 21842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:24.130635 21842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:24.133318 21842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:24.133431 21842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:24.133618 21842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:24.133862 21842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:24.133890 21842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:24.133912 21842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:24.133925 21842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:24.139593 21842 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:41689
I20260619 08:51:24.139672 21970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:41689 every 8 connection(s)
I20260619 08:51:24.139997 21842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:24.144867 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 21842
I20260619 08:51:24.144992 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:24.145198 16192 external_mini_cluster.cc:1428] Setting key 0d2b511eecb65e84abb85f8932bb97a5
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:24.150220 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:24.140719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:33283 (local address 127.15.208.1:59365)
0619 08:51:24.140936 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:24.140970 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:24.141865 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:24.142076 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:24.142079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:24.142317 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:24.142521 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:24.142525 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.143208 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.143208 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:24.144058 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:24.144060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.144394 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.144395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:24.144491 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:24.145117 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:24.145133 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:24.146656 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:24.148590 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:24.148596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:24.148598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:24.148896 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:24.149212 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:24.149213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:24.149214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:24.149321 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:24.149652 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:24.149659 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:24.149814 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 08:51:24.149897 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:51:24.150059 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260619 08:51:24.150224 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:24.140761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:59365 (local address 127.15.208.62:33283)
0619 08:51:24.140836 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:24.140838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:24.141051 (+ 213us) server_negotiation.cc:407] Connection header received
0619 08:51:24.141950 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:24.141951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:24.141965 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:24.142013 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:24.142620 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.143118 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.144130 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.144324 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.144375 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:24.146754 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:24.146756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:24.146756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:24.146765 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:24.148503 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.148964 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.148965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.148966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.149029 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.149410 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.149411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.149412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.149561 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:24.149631 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:24.149983 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 08:51:24.150091 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:51:24.150854 21971 heartbeater.cc:347] Connected to a master server at 127.15.208.62:33283
I20260619 08:51:24.150943 21971 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:24.151139 21971 heartbeater.cc:511] Master 127.15.208.62:33283 requested a full tablet report, sending...
I20260619 08:51:24.151641 21780 ts_manager.cc:194] Registered new tserver with Master: 82524820224c4a61a11c352bc276c3b3 (127.15.208.1:41689)
I20260619 08:51:24.152201 21780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:59365
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:51:24.170120 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33283
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:24.250142 21976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:24.250408 21976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:24.250445 21976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:24.251672 21976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:24.251746 21976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:24.251791 21976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:24.253340 21976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:24.253616 21976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:24.253845 21976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:24.259004 21984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:24.259032 21982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:24.259004 21981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:24.259456 21976 server_base.cc:1034] running on GCE node
I20260619 08:51:24.259645 21976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:24.259886 21976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:24.261026 21976 hybrid_clock.cc:648] HybridClock initialized: now 1781859084261006 us; error 35 us; skew 500 ppm
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:24.263432 21976 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:51:24.263902 21976 webserver.cc:460] Webserver started at http://127.15.208.2:33353/ using document root <none> and password file <none>
I20260619 08:51:24.264096 21976 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:24.264137 21976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:24.264222 21976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:24.265328 21976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "be82fbcaac794fb59bb7f9bb0591c32c"
format_stamp: "Formatted at 2026-06-19 08:51:24 on dist-test-slave-p0h9"
server_key: "e9d73d9031a457cdfe9bbf2a2a509e6a"
server_key_iv: "d188fdd44a4e09fdd214e208a580db3c"
server_key_version: "encryptionkey@0"
I20260619 08:51:24.265627 21976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "be82fbcaac794fb59bb7f9bb0591c32c"
format_stamp: "Formatted at 2026-06-19 08:51:24 on dist-test-slave-p0h9"
server_key: "e9d73d9031a457cdfe9bbf2a2a509e6a"
server_key_iv: "d188fdd44a4e09fdd214e208a580db3c"
server_key_version: "encryptionkey@0"
I20260619 08:51:24.266865 21976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:24.267537 21991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:24.267706 21976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:24.267795 21976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "be82fbcaac794fb59bb7f9bb0591c32c"
format_stamp: "Formatted at 2026-06-19 08:51:24 on dist-test-slave-p0h9"
server_key: "e9d73d9031a457cdfe9bbf2a2a509e6a"
server_key_iv: "d188fdd44a4e09fdd214e208a580db3c"
server_key_version: "encryptionkey@0"
I20260619 08:51:24.267875 21976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:24.289958 21976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:24.292047 21976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:24.292225 21976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:24.292513 21976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:24.292913 21976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:24.292982 21976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:24.293030 21976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:24.293061 21976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:24.299491 21976 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:45959
I20260619 08:51:24.299551 22104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:45959 every 8 connection(s)
I20260619 08:51:24.299930 21976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:51:24.305353 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 21976
I20260619 08:51:24.305498 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:24.305720 16192 external_mini_cluster.cc:1428] Setting key c3fd17ba1b8e7de7d4b19500007ab440
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:24.310154 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:24.300786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:44005 (local address 127.15.208.62:33283)
0619 08:51:24.300852 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:24.300854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:24.301056 (+ 202us) server_negotiation.cc:407] Connection header received
0619 08:51:24.302008 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:24.302012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:24.302022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:24.302085 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:24.302679 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.303201 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.304209 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.304410 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.304461 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:24.307201 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:24.307216 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:24.307216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:24.307222 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:24.308617 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.309108 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.309109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.309109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.309171 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.309431 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.309432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.309432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.309589 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:24.309654 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:24.309917 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 08:51:24.310010 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:51:24.310185 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:24.300725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:33283 (local address 127.15.208.2:44005)
0619 08:51:24.300908 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:24.300957 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:24.301902 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:24.302143 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:24.302147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:24.302402 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:24.302602 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:24.302606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.303293 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.303294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:24.304129 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:24.304131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.304500 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.304501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:24.304607 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:24.305167 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:24.305178 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:24.307108 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:24.308699 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:24.308706 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:24.308708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:24.309030 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:24.309257 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:24.309258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:24.309258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:24.309358 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:24.309652 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:24.309656 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:24.309770 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 08:51:24.309853 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:51:24.309977 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260619 08:51:24.310654 22105 heartbeater.cc:347] Connected to a master server at 127.15.208.62:33283
I20260619 08:51:24.310750 22105 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:24.310959 22105 heartbeater.cc:511] Master 127.15.208.62:33283 requested a full tablet report, sending...
I20260619 08:51:24.311380 21780 ts_manager.cc:194] Registered new tserver with Master: be82fbcaac794fb59bb7f9bb0591c32c (127.15.208.2:45959)
I20260619 08:51:24.311918 21780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:44005
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:51:24.331218 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33283
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:24.413990 22110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:24.414301 22110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:24.414347 22110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:24.416147 22110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:24.416213 22110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:24.416281 22110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:24.418318 22110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:33283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:24.418645 22110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:24.418974 22110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:24.424672 22116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:24.424772 22118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:24.424947 22110 server_base.cc:1034] running on GCE node
W20260619 08:51:24.424998 22115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:24.425235 22110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:24.425516 22110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:24.426689 22110 hybrid_clock.cc:648] HybridClock initialized: now 1781859084426678 us; error 34 us; skew 500 ppm
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:24.429680 22110 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:51:24.430173 22110 webserver.cc:460] Webserver started at http://127.15.208.3:35273/ using document root <none> and password file <none>
I20260619 08:51:24.430464 22110 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:24.430526 22110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:24.430672 22110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:24.431886 22110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "751ab2d9900847e69409dde4863e7ce5"
format_stamp: "Formatted at 2026-06-19 08:51:24 on dist-test-slave-p0h9"
server_key: "c0ed1d61a8abebdbfef2ab662eefb3b2"
server_key_iv: "40c9b3bd2ecc89c0274103c01d85b779"
server_key_version: "encryptionkey@0"
I20260619 08:51:24.432312 22110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "751ab2d9900847e69409dde4863e7ce5"
format_stamp: "Formatted at 2026-06-19 08:51:24 on dist-test-slave-p0h9"
server_key: "c0ed1d61a8abebdbfef2ab662eefb3b2"
server_key_iv: "40c9b3bd2ecc89c0274103c01d85b779"
server_key_version: "encryptionkey@0"
I20260619 08:51:24.433902 22110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:51:24.434800 22125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:24.435036 22110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:24.435137 22110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "751ab2d9900847e69409dde4863e7ce5"
format_stamp: "Formatted at 2026-06-19 08:51:24 on dist-test-slave-p0h9"
server_key: "c0ed1d61a8abebdbfef2ab662eefb3b2"
server_key_iv: "40c9b3bd2ecc89c0274103c01d85b779"
server_key_version: "encryptionkey@0"
I20260619 08:51:24.435215 22110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:24.452090 22110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:24.454309 22110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:24.454481 22110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:24.454748 22110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:24.455119 22110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:24.455173 22110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:24.455211 22110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:24.455237 22110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:24.461308 22110 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:44503
I20260619 08:51:24.461376 22238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:44503 every 8 connection(s)
I20260619 08:51:24.461702 22110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:24.466032 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 22110
I20260619 08:51:24.466141 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:24.466355 16192 external_mini_cluster.cc:1428] Setting key eac7374b8281c1f1d4d8814c04c59998
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:24.470963 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:24.462467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:58043 (local address 127.15.208.62:33283)
0619 08:51:24.462526 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:24.462528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:24.462743 (+ 215us) server_negotiation.cc:407] Connection header received
0619 08:51:24.463666 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:24.463667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:24.463676 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:24.463738 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:24.464342 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.464969 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.465996 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.466222 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.466300 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:24.468510 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:24.468512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:24.468512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:24.468518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:24.469773 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.470133 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.470134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.470134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.470183 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.470381 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.470381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.470381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.470480 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:24.470585 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:24.470811 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 08:51:24.470879 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:51:24.471213 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:24.462439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:33283 (local address 127.15.208.3:58043)
0619 08:51:24.462618 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:24.462661 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:24.463570 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:24.463799 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:24.463802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:24.464044 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:24.464239 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:24.464243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.465072 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.465073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:24.465911 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:24.465914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.466322 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.466324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:24.466469 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:24.467080 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:24.467088 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:24.468393 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:24.469843 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:24.469848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:24.469849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:24.470068 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:24.470238 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:24.470239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:24.470239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:24.470329 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:24.470559 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:24.470563 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:24.470670 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 08:51:24.470753 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:51:24.470872 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":125,"threads_started":1}
I20260619 08:51:24.471761 22239 heartbeater.cc:347] Connected to a master server at 127.15.208.62:33283
I20260619 08:51:24.471873 22239 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:24.472045 22239 heartbeater.cc:511] Master 127.15.208.62:33283 requested a full tablet report, sending...
I20260619 08:51:24.472473 21780 ts_manager.cc:194] Registered new tserver with Master: 751ab2d9900847e69409dde4863e7ce5 (127.15.208.3:44503)
I20260619 08:51:24.472913 21780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:58043
I20260619 08:51:24.477926 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/test-admin.keytab.
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:24.493065 16192 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 08:51:24.494942 22246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:24.494034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53744 (local address 127.15.208.1:41689)
0619 08:51:24.494230 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:24.494235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:24.494248 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:51:24.494284 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:24.494286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:24.494300 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:24.494369 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:24.494842 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53744: BlockingRecv error: recv got EOF from 127.0.0.1:53744 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
W20260619 08:51:24.495021 16192 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5cc' of type FILE
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:24.496161 16192 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:24.497252 16192 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:51:24.505699 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:24.497931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.15.208.1:41689)
0619 08:51:24.498043 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:24.498045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:24.498055 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:24.498130 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:24.498130 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:24.498140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:24.498195 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:24.498660 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.499371 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.500182 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.500488 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.500549 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:24.502713 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:24.502720 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:24.502721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:24.502727 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:24.504108 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.504645 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.504650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.504651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.504800 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.505072 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.505073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.505073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.505268 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:24.505431 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:24.505503 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 08:51:24.505583 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260619 08:51:24.507521 22246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:24.506850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.15.208.1:41689)
0619 08:51:24.506898 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:24.506900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:24.506934 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:51:24.507004 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:24.507005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:24.507016 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:24.507079 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:24.507477 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53766: BlockingRecv error: recv got EOF from 127.0.0.1:53766 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260619 08:51:24.507611 16192 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5cc' of type FILE
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:24.508586 16192 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:24.509491 16192 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:51:24.515654 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:24.509954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53770 (local address 127.15.208.1:41689)
0619 08:51:24.510005 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:24.510007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:24.510101 (+ 94us) server_negotiation.cc:407] Connection header received
0619 08:51:24.510133 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:24.510133 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:24.510141 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:24.510188 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:24.510477 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.510915 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.511539 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.511717 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.511764 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:24.513575 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:24.513577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:24.513577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:24.513582 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:24.514746 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.515062 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.515062 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.515063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.515108 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.515260 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.515261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.515261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.515340 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:24.515406 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:24.515481 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 08:51:24.515531 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260619 08:51:24.517083 22246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:24.516591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53786 (local address 127.15.208.1:41689)
0619 08:51:24.516624 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:24.516626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:24.516656 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:51:24.516700 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:24.516700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:24.516707 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:24.516813 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:24.517026 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53786: BlockingRecv error: recv got EOF from 127.0.0.1:53786 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 08:51:24.517186 16192 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781859057751076-16192-0/krb5kdc/krb5cc' of type FILE
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:24.518136 16192 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:24.518970 16192 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[21735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:51:24.525502 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:24.519424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.15.208.1:41689)
0619 08:51:24.519473 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:24.519475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:24.519562 (+ 87us) server_negotiation.cc:407] Connection header received
0619 08:51:24.519592 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:24.519593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:24.519599 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:24.519641 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:24.519932 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.520414 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.521150 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:24.521340 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:24.521400 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:24.523302 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:24.523303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:24.523303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:24.523309 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:24.524419 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.524706 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.524707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.524707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.524819 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:24.525080 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:24.525081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:24.525081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:24.525168 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:24.525238 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:24.525362 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 08:51:24.525413 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:51:24.526194 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 21842
I20260619 08:51:24.531648 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 21976
I20260619 08:51:24.535792 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 22110
I20260619 08:51:24.540179 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 21748
2026-06-19T08:51:24Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3626 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[22259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[22259](info): set up 2 sockets
Jun 19 08:51:24 dist-test-slave-p0h9 krb5kdc[22259](info): commencing operation
krb5kdc: starting...
W20260619 08:51:25.577446 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:25 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.62.keytab.
Entry for principal oryx/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.62.keytab.
I20260619 08:51:25.695052 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46545
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.62.keytab
--principal=oryx/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:46545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:25.775733 22272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:25.775949 22272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:25.775971 22272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:25.777179 22272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:25.777222 22272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:25.777236 22272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:25.777249 22272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:25.777290 22272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:25.778779 22272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:46545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.62.keytab
--principal=oryx/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:25.779028 22272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:25.779259 22272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:25.784277 22280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:25.784309 22278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:25.784286 22277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:25.784317 22272 server_base.cc:1034] running on GCE node
I20260619 08:51:25.784765 22272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:25.785056 22272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:25.786213 22272 hybrid_clock.cc:648] HybridClock initialized: now 1781859085786203 us; error 28 us; skew 500 ppm
Jun 19 08:51:25 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859085, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:25.788599 22272 init.cc:377] Logged in from keytab as oryx/127.15.208.62@KRBTEST.COM (short username oryx)
I20260619 08:51:25.789063 22272 webserver.cc:460] Webserver started at http://127.15.208.62:39741/ using document root <none> and password file <none>
I20260619 08:51:25.789283 22272 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:25.789351 22272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:25.789516 22272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:25.790625 22272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "da6a78a0b5bb473aa819bb6ce926b5b2"
format_stamp: "Formatted at 2026-06-19 08:51:25 on dist-test-slave-p0h9"
server_key: "044a0150c619f2d0656e5210cdb023a1"
server_key_iv: "bbe1b8bf1ce2329592192a5b34ef17cd"
server_key_version: "encryptionkey@0"
I20260619 08:51:25.790993 22272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "da6a78a0b5bb473aa819bb6ce926b5b2"
format_stamp: "Formatted at 2026-06-19 08:51:25 on dist-test-slave-p0h9"
server_key: "044a0150c619f2d0656e5210cdb023a1"
server_key_iv: "bbe1b8bf1ce2329592192a5b34ef17cd"
server_key_version: "encryptionkey@0"
I20260619 08:51:25.792351 22272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:25.793125 22287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:25.793296 22272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:25.793403 22272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "da6a78a0b5bb473aa819bb6ce926b5b2"
format_stamp: "Formatted at 2026-06-19 08:51:25 on dist-test-slave-p0h9"
server_key: "044a0150c619f2d0656e5210cdb023a1"
server_key_iv: "bbe1b8bf1ce2329592192a5b34ef17cd"
server_key_version: "encryptionkey@0"
I20260619 08:51:25.793476 22272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:25.817544 22272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:25.819473 22272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:25.819633 22272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:25.824476 22272 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:46545
I20260619 08:51:25.824532 22339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:46545 every 8 connection(s)
I20260619 08:51:25.824934 22272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:25.825925 22340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:25.828517 22340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: Bootstrap starting.
I20260619 08:51:25.829182 22340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:25.829481 22340 log.cc:824] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:25.830106 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 22272
I20260619 08:51:25.830195 22340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: No bootstrap required, opened a new log
I20260619 08:51:25.830188 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:25.830401 16192 external_mini_cluster.cc:1428] Setting key 2e602b7aec33d8fa4f44783ae79a098b
I20260619 08:51:25.832472 22340 raft_consensus.cc:348] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } }
I20260619 08:51:25.832652 22340 raft_consensus.cc:374] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:25.832696 22340 raft_consensus.cc:729] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6a78a0b5bb473aa819bb6ce926b5b2, State: Initialized, Role: FOLLOWER
I20260619 08:51:25.832868 22340 consensus_queue.cc:260] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } }
I20260619 08:51:25.832945 22340 raft_consensus.cc:388] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:25.832975 22340 raft_consensus.cc:482] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:25.833017 22340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:25.833711 22340 raft_consensus.cc:504] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } }
I20260619 08:51:25.833814 22340 leader_election.cc:302] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da6a78a0b5bb473aa819bb6ce926b5b2; no voters:
I20260619 08:51:25.833963 22340 leader_election.cc:288] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:25.834028 22345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:25.834165 22345 raft_consensus.cc:686] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 1 LEADER]: Becoming Leader. State: Replica: da6a78a0b5bb473aa819bb6ce926b5b2, State: Running, Role: LEADER
I20260619 08:51:25.834177 22340 sys_catalog.cc:564] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:25.834290 22345 consensus_queue.cc:237] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } }
I20260619 08:51:25.836335 22347 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } }
I20260619 08:51:25.836301 22346 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da6a78a0b5bb473aa819bb6ce926b5b2. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } }
I20260619 08:51:25.836424 22347 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:25.836433 22346 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: This master's current role is: LEADER
W20260619 08:51:25.836884 22359 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:51:25.836934 22359 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:51:25.837036 22360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:25.837162 22360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:51:25 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:25.838644 22360 catalog_manager.cc:1347] Generated new cluster ID: 0defa86d538b4ccc9c8e6e52a435bad3
I20260619 08:51:25.838701 22360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:25.842588 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:25.830940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51464 (local address 127.15.208.62:46545)
0619 08:51:25.831157 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:25.831166 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:25.831199 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:51:25.831693 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:25.831702 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:25.831761 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:25.831908 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:25.832478 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:25.834161 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:25.835117 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:25.835718 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:25.835820 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:25.838919 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:25.838923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:25.838924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:25.838936 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:25.840961 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:25.841357 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:25.841360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:25.841361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:25.841469 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:25.841676 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:25.841677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:25.841678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:25.841985 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:25.842155 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:25.842235 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 08:51:25.842370 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":108,"threads_started":1}
I20260619 08:51:25.853024 22360 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:25.853565 22360 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:25.857357 22360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: Generated new TSK 0
I20260619 08:51:25.857538 22360 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab.
Entry for principal oryx/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab.
I20260619 08:51:25.885775 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46545
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab
--principal=oryx/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:25.967661 22366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:25.967862 22366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:25.967886 22366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:25.969125 22366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:25.969165 22366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:25.969192 22366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:25.970715 22366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab
--principal=oryx/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:25.970965 22366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:25.971208 22366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:25.976327 22374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:25.976450 22366 server_base.cc:1034] running on GCE node
W20260619 08:51:25.976356 22372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:25.976596 22371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:25.976853 22366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:25.977077 22366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:25.978217 22366 hybrid_clock.cc:648] HybridClock initialized: now 1781859085978199 us; error 34 us; skew 500 ppm
Jun 19 08:51:25 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859085, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:25.980711 22366 init.cc:377] Logged in from keytab as oryx/127.15.208.1@KRBTEST.COM (short username oryx)
I20260619 08:51:25.981317 22366 webserver.cc:460] Webserver started at http://127.15.208.1:46483/ using document root <none> and password file <none>
I20260619 08:51:25.981545 22366 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:25.981617 22366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:25.981772 22366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:25.982824 22366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "851148a914324cc5ba28a76da4424c45"
format_stamp: "Formatted at 2026-06-19 08:51:25 on dist-test-slave-p0h9"
server_key: "347b57c2b826c1f1c1e00a6f896072dc"
server_key_iv: "ea99a3b171b6fc5d5d1365e513eaf437"
server_key_version: "encryptionkey@0"
I20260619 08:51:25.983229 22366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "851148a914324cc5ba28a76da4424c45"
format_stamp: "Formatted at 2026-06-19 08:51:25 on dist-test-slave-p0h9"
server_key: "347b57c2b826c1f1c1e00a6f896072dc"
server_key_iv: "ea99a3b171b6fc5d5d1365e513eaf437"
server_key_version: "encryptionkey@0"
I20260619 08:51:25.984594 22366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:25.985383 22381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:25.985533 22366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:25.985639 22366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "851148a914324cc5ba28a76da4424c45"
format_stamp: "Formatted at 2026-06-19 08:51:25 on dist-test-slave-p0h9"
server_key: "347b57c2b826c1f1c1e00a6f896072dc"
server_key_iv: "ea99a3b171b6fc5d5d1365e513eaf437"
server_key_version: "encryptionkey@0"
I20260619 08:51:25.985711 22366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:26.005692 22366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:26.008131 22366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:26.008298 22366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:26.008553 22366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:26.008934 22366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:26.008989 22366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.009032 22366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:26.009076 22366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.015103 22366 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:36031
I20260619 08:51:26.015146 22494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:36031 every 8 connection(s)
I20260619 08:51:26.015506 22366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:26.020684 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 22366
I20260619 08:51:26.020817 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:26.021049 16192 external_mini_cluster.cc:1428] Setting key 1e517de8920cebdbebca2045a34a58f6
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859085, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:26.025017 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.016235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:46073 (local address 127.15.208.62:46545)
0619 08:51:26.016280 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.016282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.016488 (+ 206us) server_negotiation.cc:407] Connection header received
0619 08:51:26.016998 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.016999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.017007 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.017077 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.018020 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.018523 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.019457 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.019648 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.019697 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.021976 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.021977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.021978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.021985 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.023539 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.024031 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.024031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.024032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.024090 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.024369 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.024370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.024370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.024469 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.024519 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.024872 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.024951 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:51:26.025311 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.016206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.1:46073)
0619 08:51:26.016371 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.016413 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.016907 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.017139 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.017142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.017714 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.017938 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.017949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.018600 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.018601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.019384 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.019385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.019716 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.019717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.019811 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.020278 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.020286 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.021888 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.023625 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.023632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.023634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.023956 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.024154 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.024154 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.024155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.024303 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.024537 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.024543 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.024712 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.024852 (+ 140us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.025030 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260619 08:51:26.025863 22495 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46545
I20260619 08:51:26.025970 22495 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:26.026161 22495 heartbeater.cc:511] Master 127.15.208.62:46545 requested a full tablet report, sending...
I20260619 08:51:26.026660 22304 ts_manager.cc:194] Registered new tserver with Master: 851148a914324cc5ba28a76da4424c45 (127.15.208.1:36031)
I20260619 08:51:26.027272 22304 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:46073
WARNING: no policy specified for oryx/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab.
Entry for principal oryx/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab.
I20260619 08:51:26.047581 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46545
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab
--principal=oryx/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:26.130909 22500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:26.131120 22500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:26.131141 22500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:26.132293 22500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:26.132332 22500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:26.132359 22500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:26.133729 22500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab
--principal=oryx/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:26.133910 22500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:26.134081 22500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:26.139065 22508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.139111 22506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.139129 22505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:26.139328 22500 server_base.cc:1034] running on GCE node
I20260619 08:51:26.139571 22500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:26.139765 22500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:26.140894 22500 hybrid_clock.cc:648] HybridClock initialized: now 1781859086140875 us; error 35 us; skew 500 ppm
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:26.143850 22500 init.cc:377] Logged in from keytab as oryx/127.15.208.2@KRBTEST.COM (short username oryx)
I20260619 08:51:26.144281 22500 webserver.cc:460] Webserver started at http://127.15.208.2:45761/ using document root <none> and password file <none>
I20260619 08:51:26.144518 22500 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:26.144568 22500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:26.144771 22500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:26.145814 22500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "bc40bdaed75848d2b8a9d58ec180887a"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "4a4922fa78b37548196bdcf1b0cbb7a5"
server_key_iv: "72e4b09c0e73fd382e69eed191c629d1"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.146206 22500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "bc40bdaed75848d2b8a9d58ec180887a"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "4a4922fa78b37548196bdcf1b0cbb7a5"
server_key_iv: "72e4b09c0e73fd382e69eed191c629d1"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.147526 22500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:26.148311 22515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.148546 22500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:26.148643 22500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "bc40bdaed75848d2b8a9d58ec180887a"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "4a4922fa78b37548196bdcf1b0cbb7a5"
server_key_iv: "72e4b09c0e73fd382e69eed191c629d1"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.148703 22500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:26.167093 22500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:26.169139 22500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:26.169263 22500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:26.169507 22500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:26.169797 22500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:26.169832 22500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.169858 22500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:26.169873 22500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.176195 22500 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:46255
I20260619 08:51:26.176254 22628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:46255 every 8 connection(s)
I20260619 08:51:26.176648 22500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:51:26.182744 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 22500
I20260619 08:51:26.182852 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:26.183104 16192 external_mini_cluster.cc:1428] Setting key 606308d052995f623341f6db9ae19d8f
I20260619 08:51:26.186980 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.177496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:52623 (local address 127.15.208.62:46545)
0619 08:51:26.177559 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.177561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.177774 (+ 213us) server_negotiation.cc:407] Connection header received
0619 08:51:26.178326 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.178328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.178340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.178381 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.179353 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.179855 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.180935 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.181135 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.181208 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.183521 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.183523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.183523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.183532 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.185329 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.185878 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.185879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.185879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.185947 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.186242 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.186243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.186243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.186360 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.186468 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.186782 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.186890 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:51:26.187150 22631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.177462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.2:52623)
0619 08:51:26.177655 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.177691 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.178207 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.178465 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.178469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.179050 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.179241 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.179245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.179951 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.179952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.180840 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.180842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.181226 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.181228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.181364 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.181873 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.181881 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.183379 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.185414 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.185438 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.185440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.185793 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.186020 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.186021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.186022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.186166 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.186432 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.186437 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.186618 (+ 181us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.186755 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.186913 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260619 08:51:26.187590 22629 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46545
I20260619 08:51:26.187711 22629 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:26.187932 22629 heartbeater.cc:511] Master 127.15.208.62:46545 requested a full tablet report, sending...
I20260619 08:51:26.188362 22304 ts_manager.cc:194] Registered new tserver with Master: bc40bdaed75848d2b8a9d58ec180887a (127.15.208.2:46255)
I20260619 08:51:26.188804 22304 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:52623
WARNING: no policy specified for oryx/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab.
Entry for principal oryx/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab.
I20260619 08:51:26.209488 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46545
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab
--principal=oryx/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:26.290380 22634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:26.290597 22634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:26.290620 22634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:26.291757 22634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:26.291801 22634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:26.291827 22634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:26.293313 22634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab
--principal=oryx/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:26.293587 22634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:26.293810 22634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:26.298779 22639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.298812 22642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.298815 22640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:26.299171 22634 server_base.cc:1034] running on GCE node
I20260619 08:51:26.299397 22634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:26.299618 22634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:26.300824 22634 hybrid_clock.cc:648] HybridClock initialized: now 1781859086300812 us; error 41 us; skew 500 ppm
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:26.303382 22634 init.cc:377] Logged in from keytab as oryx/127.15.208.3@KRBTEST.COM (short username oryx)
I20260619 08:51:26.303934 22634 webserver.cc:460] Webserver started at http://127.15.208.3:33771/ using document root <none> and password file <none>
I20260619 08:51:26.304167 22634 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:26.304237 22634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:26.304399 22634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:26.305673 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "e8768e3237994432af741ad63b27d8d3"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "307b7e9e29579682147e5f638064f8f6"
server_key_iv: "9a2320c379162d70babb349dddb43658"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.306072 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "e8768e3237994432af741ad63b27d8d3"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "307b7e9e29579682147e5f638064f8f6"
server_key_iv: "9a2320c379162d70babb349dddb43658"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.307533 22634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:51:26.308247 22649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.308457 22634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:26.308569 22634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "e8768e3237994432af741ad63b27d8d3"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "307b7e9e29579682147e5f638064f8f6"
server_key_iv: "9a2320c379162d70babb349dddb43658"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.308658 22634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:26.321149 22634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:26.323100 22634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:26.323300 22634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:26.323567 22634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:26.323913 22634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:26.323976 22634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.324016 22634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:26.324070 22634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.330606 22634 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:45669
I20260619 08:51:26.330652 22762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:45669 every 8 connection(s)
I20260619 08:51:26.331002 22634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:26.334086 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 22634
I20260619 08:51:26.334178 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:26.334422 16192 external_mini_cluster.cc:1428] Setting key 1a5154b4037dbca83e547549aa4ed2dc
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:26.341228 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.331738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:34263 (local address 127.15.208.62:46545)
0619 08:51:26.331802 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.331804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.332069 (+ 265us) server_negotiation.cc:407] Connection header received
0619 08:51:26.332614 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.332616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.332633 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.332687 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.333698 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.334417 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.335915 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.336115 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.336171 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.338577 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.338579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.338579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.338586 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.339972 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.340342 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.340343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.340343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.340392 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.340624 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.340624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.340625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.340759 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.340848 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.341102 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.341164 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:51:26.341406 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.331704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.3:34263)
0619 08:51:26.331895 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.331956 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.332481 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.332773 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.332777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.333366 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.333589 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.333594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.334547 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.334548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.335842 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.335845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.336205 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.336206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.336444 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.337030 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.337039 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.338485 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.340057 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.340062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.340064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.340275 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.340448 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.340448 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.340449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.340569 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.340845 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.340857 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.340980 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.341074 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.341196 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260619 08:51:26.341871 22763 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46545
I20260619 08:51:26.341993 22763 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:26.342171 22763 heartbeater.cc:511] Master 127.15.208.62:46545 requested a full tablet report, sending...
I20260619 08:51:26.342610 22304 ts_manager.cc:194] Registered new tserver with Master: e8768e3237994432af741ad63b27d8d3 (127.15.208.3:45669)
I20260619 08:51:26.342984 22304 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:34263
I20260619 08:51:26.346266 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.208.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.61.keytab.
Entry for principal oryx/127.15.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.61.keytab.
I20260619 08:51:26.369701 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.61:34855
--webserver_interface=127.15.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.61.keytab
--principal=oryx/127.15.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:46545,127.15.208.61:34855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:26.451853 22768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:26.452080 22768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:26.452116 22768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:26.453300 22768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:26.453374 22768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:26.453399 22768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:26.453428 22768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:26.453466 22768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:26.455284 22768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:46545,127.15.208.61:34855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.61.keytab
--principal=oryx/127.15.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.61:34855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
W20260619 08:51:26.455413 22768 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.208.62:46545,127.15.208.61:34855'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 08:51:26.455579 22768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:26.455806 22768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:26.460786 22773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.460763 22776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.460795 22774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:26.461189 22768 server_base.cc:1034] running on GCE node
I20260619 08:51:26.461560 22768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:26.461860 22768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:26.463034 22768 hybrid_clock.cc:648] HybridClock initialized: now 1781859086463030 us; error 42 us; skew 500 ppm
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:26.465808 22768 init.cc:377] Logged in from keytab as oryx/127.15.208.61@KRBTEST.COM (short username oryx)
I20260619 08:51:26.466261 22768 webserver.cc:460] Webserver started at http://127.15.208.61:39329/ using document root <none> and password file <none>
I20260619 08:51:26.466454 22768 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:26.466504 22768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:26.466596 22768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:26.467612 22768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data/instance:
uuid: "0806610f714f4ed295d35650ab23e71d"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "73f23f3e2fff950d868632795abefd45"
server_key_iv: "48be69173876ce38f059df68fb290835"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.467912 22768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal/instance:
uuid: "0806610f714f4ed295d35650ab23e71d"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "73f23f3e2fff950d868632795abefd45"
server_key_iv: "48be69173876ce38f059df68fb290835"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.469213 22768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 08:51:26.469931 22783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.470111 22768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:26.470197 22768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal
uuid: "0806610f714f4ed295d35650ab23e71d"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "73f23f3e2fff950d868632795abefd45"
server_key_iv: "48be69173876ce38f059df68fb290835"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.470248 22768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:26.490402 22768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:26.492347 22768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:26.492519 22768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:26.496795 22768 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.61:34855
I20260619 08:51:26.496824 22835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.61:34855 every 8 connection(s)
I20260619 08:51:26.497223 22768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
I20260619 08:51:26.498294 22836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:26.499775 22836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.61@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:26.505357 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 22768
I20260619 08:51:26.508901 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.500453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51474 (local address 127.15.208.62:46545)
0619 08:51:26.500510 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.500512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.500750 (+ 238us) server_negotiation.cc:407] Connection header received
0619 08:51:26.501256 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.501258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.501271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.501321 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.501918 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.502415 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.503390 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.503594 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.503679 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.505854 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.505856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.505856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.505864 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.507412 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.507853 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.507854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.507855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.507904 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.508169 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.508170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.508171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.508364 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.508438 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.508694 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.508806 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:51:26.508988 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.500409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:51474)
0619 08:51:26.500616 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.500650 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.501141 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.501392 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.501396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.501640 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.501824 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.501828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.502505 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.502506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.503300 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.503302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.503687 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.503688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.503804 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.504312 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.504319 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.505751 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.507488 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.507492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.507494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.507781 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.507964 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.507965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.507965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.508099 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.508433 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.508437 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.508543 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.508641 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.508809 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":121,"threads_started":1}
I20260619 08:51:26.509578 22836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.208.61@KRBTEST.COM
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.61@KRBTEST.COM for oryx/127.15.208.61@KRBTEST.COM
I20260619 08:51:26.515379 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.505918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43292 (local address 127.15.208.61:34855)
0619 08:51:26.506042 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.506047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.506152 (+ 105us) server_negotiation.cc:407] Connection header received
0619 08:51:26.506178 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.506180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.506192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.506270 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.507093 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.508000 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.509229 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.509601 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.509655 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.512143 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.512149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.512149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.512154 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.514079 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.514461 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.514465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.514465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.514560 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.514753 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.514754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.514754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.514958 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.515138 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.515183 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.515257 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":75,"threads_started":1}
I20260619 08:51:26.516968 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.509830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34855 (local address 127.0.0.1:43298)
0619 08:51:26.509874 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.509884 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.509918 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.510161 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.510161 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.510300 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.510335 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.510336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.510966 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.510967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.511723 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.511725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.512099 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.512101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.512175 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.512563 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.512567 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.513732 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.516100 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.516100 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.516101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.516277 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.516468 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.516468 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.516469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.516517 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.516712 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.516713 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.516788 (+ 75us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.516841 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.516912 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 08:51:26.517127 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.509893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.15.208.61:34855)
0619 08:51:26.510046 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.510048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.510054 (+ 6us) server_negotiation.cc:407] Connection header received
0619 08:51:26.510070 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.510070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.510077 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.510146 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.510411 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.510863 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.511794 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.512015 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.512075 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.513811 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.513812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.513812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.513819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.516011 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.516358 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.516359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.516360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.516410 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.516573 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.516574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.516574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.516661 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.516706 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.516859 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.516956 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260619 08:51:26.518361 22836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: Bootstrap starting.
I20260619 08:51:26.518949 22836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:26.519223 22836 log.cc:824] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:26.519989 22836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: No bootstrap required, opened a new log
I20260619 08:51:26.521436 22836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } }
I20260619 08:51:26.521555 22836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:26.521577 22836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0806610f714f4ed295d35650ab23e71d, State: Initialized, Role: FOLLOWER
I20260619 08:51:26.521714 22836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } }
I20260619 08:51:26.521962 22843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } } }
I20260619 08:51:26.522006 22836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:26.522073 22843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:51:26.523840 22854 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:51:26.523917 22854 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:51:26.525416 22768 master_runner.cc:428] Detected that this master 0806610f714f4ed295d35650ab23e71d is joining an existing cluster
I20260619 08:51:26.525465 22768 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.208.61:34855
I20260619 08:51:26.532552 22861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.526305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:51480)
0619 08:51:26.526499 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.526512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.526574 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.526746 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.526747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.526951 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.527013 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.527015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.527681 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.527682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.528231 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.528233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.528618 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.528619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.528686 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.529214 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.529218 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.529561 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.531189 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.531190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.531190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.531401 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.531642 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.531643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.531644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.531703 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.532148 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.532149 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.532192 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.532326 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.532440 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":66,"threads_started":1}
I20260619 08:51:26.532558 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.526370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51480 (local address 127.15.208.62:46545)
0619 08:51:26.526448 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.526450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.526562 (+ 112us) server_negotiation.cc:407] Connection header received
0619 08:51:26.526639 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.526640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.526651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.526682 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.527100 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.527600 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.528313 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.528526 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.528618 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.529652 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.529653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.529654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.529661 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.531108 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.531506 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.531507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.531508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.531569 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.531955 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.531956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.531957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.532073 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.532157 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.532373 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.532455 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 08:51:26.535671 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.526516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.15.208.61:34855)
0619 08:51:26.526552 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.526554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.526563 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:26.526633 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.526634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.526643 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.526698 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.527150 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.527879 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.528937 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.529223 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.529262 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.531541 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.531542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.531542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.531548 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.533543 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.534415 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.534416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.534416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.534491 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.535115 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.535116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.535117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.535259 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.535318 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.535486 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.535580 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:51:26.536134 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.526355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34855 (local address 127.0.0.1:43306)
0619 08:51:26.526506 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.526516 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.526569 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.526717 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.526718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.527001 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.527064 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.527065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.527963 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.527964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.528851 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.528853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.529326 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.529326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.529380 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.529935 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.529940 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.531446 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.533887 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.533888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.533889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.534289 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.534586 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.534587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.534587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.535039 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.535338 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.535339 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.535386 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.535932 (+ 546us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.536036 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859085, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.62@KRBTEST.COM for oryx/127.15.208.61@KRBTEST.COM
I20260619 08:51:26.542474 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.534863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34855 (local address 127.0.0.1:43316)
0619 08:51:26.535012 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.535043 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.535150 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.535309 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.535313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.535679 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.535741 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.535744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.536678 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.536679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.537311 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.537314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.537814 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.537815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.537880 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.538336 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.538341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.539844 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.541509 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.541516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.541517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.541704 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.541900 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.541900 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.541901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.541952 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.542202 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.542207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.542248 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.542311 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.542396 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260619 08:51:26.542523 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.534896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.15.208.61:34855)
0619 08:51:26.534931 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.534933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.535113 (+ 180us) server_negotiation.cc:407] Connection header received
0619 08:51:26.535241 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.535242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.535252 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.535305 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.535841 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.536570 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.537395 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.537735 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.537807 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.539928 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.539929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.539929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.539936 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.541407 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.541777 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.541778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.541778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.541833 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.542011 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.542012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.542012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.542139 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.542207 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.542353 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.542429 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:51:26.543081 22304 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.208.61:34855: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0806610f714f4ed295d35650ab23e71d"
member_type: NON_VOTER
last_known_addr {
host: "127.15.208.61"
port: 34855
}
attrs {
promote: true
}
}
dest_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2"
cas_config_opid_index: -1
I20260619 08:51:26.543412 22304 consensus_queue.cc:237] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } }
I20260619 08:51:26.543862 22864 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0806610f714f4ed295d35650ab23e71d (127.15.208.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } } }
I20260619 08:51:26.544145 22810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:26.544806 22864 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.208.61:34855
I20260619 08:51:26.544950 22810 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 FOLLOWER]: Refusing update from remote peer da6a78a0b5bb473aa819bb6ce926b5b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 08:51:26.545045 22768 master.cc:537] Master@127.15.208.61:34855 shutting down...
I20260619 08:51:26.545028 22863 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } } }
I20260619 08:51:26.545253 22863 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:26.545411 22863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 08:51:26.546226 22290 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 08:51:26.546789 22290 consensus_peers.cc:487] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 -> Peer 0806610f714f4ed295d35650ab23e71d (127.15.208.61:34855): Couldn't send request to peer 0806610f714f4ed295d35650ab23e71d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:51:26.547912 22768 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:26.547996 22768 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:26.548046 22768 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: stopping tablet replica
I20260619 08:51:26.560689 22768 master.cc:552] Master@127.15.208.61:34855 shutdown complete.
I20260619 08:51:26.560812 22768 master_runner.cc:305] Clearing existing system tablet
I20260619 08:51:26.560945 22768 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:26.561007 22768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:26.561609 22768 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:26.562075 22868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.562191 22768 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:26.562248 22768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.562270 22768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal
uuid: "0806610f714f4ed295d35650ab23e71d"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "73f23f3e2fff950d868632795abefd45"
server_key_iv: "48be69173876ce38f059df68fb290835"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.562930 22768 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:26.564325 22768 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 08:51:26.564383 22768 log.cc:1196] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 08:51:26.564610 22768 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: Deleting consensus metadata
I20260619 08:51:26.564770 22768 master_runner.cc:315] Copying system tablet from 127.15.208.62:46545
I20260619 08:51:26.565423 22768 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: tablet copy: Beginning tablet copy session from remote peer at address 127.15.208.62:46545
I20260619 08:51:26.571812 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.565817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.15.208.62:46545)
0619 08:51:26.565874 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.565877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.566003 (+ 126us) server_negotiation.cc:407] Connection header received
0619 08:51:26.566096 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.566097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.566107 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.566166 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.566596 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.567097 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.567841 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.568052 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.568138 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.569210 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.569211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.569211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.569218 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.570651 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.571030 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.571031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.571032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.571088 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.571295 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.571296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.571296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.571410 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.571485 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.571643 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.571716 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:51:26.571827 22874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.565753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:51486)
0619 08:51:26.565870 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.565882 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.566021 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.566170 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.566171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.566387 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.566465 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.566473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.567184 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.567185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.567750 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.567752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.568140 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.568141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.568207 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.568701 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.568706 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.569100 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.570759 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.570760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.570761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.570955 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.571162 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.571163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.571164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.571219 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.571477 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.571478 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.571521 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.571612 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.571717 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":72,"threads_started":1}
I20260619 08:51:26.572147 22324 tablet_copy_service.cc:140] P da6a78a0b5bb473aa819bb6ce926b5b2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0806610f714f4ed295d35650ab23e71d ({username='oryx', principal='oryx/127.15.208.61@KRBTEST.COM'} at 127.0.0.1:51486)
I20260619 08:51:26.572242 22324 tablet_copy_service.cc:161] P da6a78a0b5bb473aa819bb6ce926b5b2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0806610f714f4ed295d35650ab23e71d at {username='oryx', principal='oryx/127.15.208.61@KRBTEST.COM'} at 127.0.0.1:51486: session id = 0806610f714f4ed295d35650ab23e71d-00000000000000000000000000000000
I20260619 08:51:26.573143 22324 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 08:51:26.573770 22768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 08:51:26.574671 22768 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: tablet copy: Starting download of 0 data blocks...
I20260619 08:51:26.574812 22768 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: tablet copy: Starting download of 1 WAL segments...
I20260619 08:51:26.575662 22768 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 08:51:26.576537 22324 tablet_copy_service.cc:342] P da6a78a0b5bb473aa819bb6ce926b5b2: Request end of tablet copy session 0806610f714f4ed295d35650ab23e71d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.208.61@KRBTEST.COM'} at 127.0.0.1:51486
I20260619 08:51:26.576603 22324 tablet_copy_service.cc:433] P da6a78a0b5bb473aa819bb6ce926b5b2: ending tablet copy session 0806610f714f4ed295d35650ab23e71d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0806610f714f4ed295d35650ab23e71d
I20260619 08:51:26.578984 22768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:26.580122 22875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.580178 22878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.580369 22876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:26.580384 22768 server_base.cc:1034] running on GCE node
I20260619 08:51:26.580510 22768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:26.580650 22768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:26.581773 22768 hybrid_clock.cc:648] HybridClock initialized: now 1781859086581764 us; error 27 us; skew 500 ppm
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:26.583194 22768 init.cc:377] Logged in from keytab as oryx/127.15.208.61@KRBTEST.COM (short username oryx)
I20260619 08:51:26.583496 22768 webserver.cc:460] Webserver started at http://127.15.208.61:44255/ using document root <none> and password file <none>
I20260619 08:51:26.583616 22768 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:26.583652 22768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:26.584249 22768 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:26.584791 22885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.584931 22768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.584992 22768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal
uuid: "0806610f714f4ed295d35650ab23e71d"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "73f23f3e2fff950d868632795abefd45"
server_key_iv: "48be69173876ce38f059df68fb290835"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.585053 22768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:26.603679 22768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:26.605346 22768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:26.608582 22768 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.61:34855
I20260619 08:51:26.608826 22937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.61:34855 every 8 connection(s)
I20260619 08:51:26.609035 22768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
I20260619 08:51:26.609752 22938 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:51:26.610194 22938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: Bootstrap starting.
I20260619 08:51:26.612449 22938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:51:26.612670 22938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: Bootstrap complete.
I20260619 08:51:26.612906 22938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } }
I20260619 08:51:26.612980 22938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0806610f714f4ed295d35650ab23e71d, State: Initialized, Role: LEARNER
I20260619 08:51:26.613013 22938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } }
I20260619 08:51:26.613184 22938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:26.613188 22941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } } }
I20260619 08:51:26.613312 22941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: This master's current role is: LEARNER
I20260619 08:51:26.614670 22952 catalog_manager.cc:1259] Loaded cluster ID: 0defa86d538b4ccc9c8e6e52a435bad3
I20260619 08:51:26.614732 22952 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: loading cluster ID for follower catalog manager: success
I20260619 08:51:26.615667 22952 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: acquiring CA information for follower catalog manager: success
I20260619 08:51:26.615935 22952 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:51:26.623932 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.616970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43330 (local address 127.15.208.61:34855)
0619 08:51:26.617080 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.617082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.617127 (+ 45us) server_negotiation.cc:407] Connection header received
0619 08:51:26.617205 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.617206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.617214 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.617272 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.617931 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.618578 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.619626 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.619815 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.619866 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.621333 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.621334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.621335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.621341 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.622794 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.623199 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.623200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.623200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.623250 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.623458 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.623458 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.623459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.623574 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.623645 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.623803 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.623864 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":53,"threads_started":1}
WARNING: no policy specified for oryx/127.15.208.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.60.keytab.
Entry for principal oryx/127.15.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.60.keytab.
I20260619 08:51:26.652022 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.60:35615
--webserver_interface=127.15.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.60.keytab
--principal=oryx/127.15.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:46545,127.15.208.61:34855,127.15.208.60:35615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:26.746902 22956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:26.747107 22956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:26.747130 22956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:26.748191 22956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:26.748227 22956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:26.748242 22956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:26.748255 22956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:26.748266 22956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:26.749680 22956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:46545,127.15.208.61:34855,127.15.208.60:35615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.60.keytab
--principal=oryx/127.15.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.60:35615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:26.749864 22956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:26.750038 22956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:26.754882 22961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:26.755049 22956 server_base.cc:1034] running on GCE node
W20260619 08:51:26.754858 22962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.754856 22964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:26.755542 22956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:26.755841 22956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:26.756995 22956 hybrid_clock.cc:648] HybridClock initialized: now 1781859086756973 us; error 36 us; skew 500 ppm
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:26.759418 22956 init.cc:377] Logged in from keytab as oryx/127.15.208.60@KRBTEST.COM (short username oryx)
I20260619 08:51:26.759889 22956 webserver.cc:460] Webserver started at http://127.15.208.60:38923/ using document root <none> and password file <none>
I20260619 08:51:26.760118 22956 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:26.760187 22956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:26.760347 22956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:26.761457 22956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data/instance:
uuid: "a88ac38d150b425c9c1ddd9fe524fa12"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "19a62e40cdc3171f4cbea3629c51a834"
server_key_iv: "f6628f78e83d00a58068faf3f3d90dad"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.761817 22956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal/instance:
uuid: "a88ac38d150b425c9c1ddd9fe524fa12"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "19a62e40cdc3171f4cbea3629c51a834"
server_key_iv: "f6628f78e83d00a58068faf3f3d90dad"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.763126 22956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:26.763830 22971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.763985 22956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:26.764097 22956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal
uuid: "a88ac38d150b425c9c1ddd9fe524fa12"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "19a62e40cdc3171f4cbea3629c51a834"
server_key_iv: "f6628f78e83d00a58068faf3f3d90dad"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.764168 22956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:26.774772 22956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:26.776609 22956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:26.776810 22956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:26.780555 22956 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.60:35615
I20260619 08:51:26.780601 23023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.60:35615 every 8 connection(s)
I20260619 08:51:26.781004 22956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
I20260619 08:51:26.782016 23024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:26.783587 23024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:51:26.787475 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 22956
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.60@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.208.60@KRBTEST.COM
I20260619 08:51:26.793817 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.784415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51490 (local address 127.15.208.62:46545)
0619 08:51:26.784483 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.784485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.784642 (+ 157us) server_negotiation.cc:407] Connection header received
0619 08:51:26.785214 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.785215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.785225 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.785287 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.785902 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.786464 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.787556 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.787826 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.787898 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.790369 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.790371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.790371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.790378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.792042 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.792645 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.792646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.792646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.792702 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.793146 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.793147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.793148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.793292 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.793379 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.793627 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.793706 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:51:26.793956 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.784266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:51490)
0619 08:51:26.784504 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.784567 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.785128 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.785355 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.785358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.785627 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.785814 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.785818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.786538 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.786539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.787448 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.787451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.787944 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.787945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.788090 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.788691 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.788699 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.790234 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.792117 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.792122 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.792124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.792421 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.792883 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.792884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.792884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.793071 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.793371 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.793376 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.793487 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.793582 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.793693 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":103,"threads_started":1}
I20260619 08:51:26.794550 23024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:51:26.797878 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.788048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.15.208.60:35615)
0619 08:51:26.788217 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.788228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.788241 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:51:26.788258 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.788259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.788290 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.788372 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.789123 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.789851 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.790778 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.791336 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.791403 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.794305 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.794311 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.794311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.794317 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.796245 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.796674 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.796677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.796678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.796773 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.797263 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.797264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.797264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.797467 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.797595 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.797721 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.797791 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":114,"threads_started":1}
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.60@KRBTEST.COM for oryx/127.15.208.61@KRBTEST.COM
I20260619 08:51:26.801410 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.794792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34855 (local address 127.0.0.1:43342)
0619 08:51:26.794831 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.794841 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.794892 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.795018 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.795019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.795189 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.795249 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.795251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.795974 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.795975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.796695 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.796697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.797106 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.797107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.797161 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.797889 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.797894 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.799161 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.800541 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.800541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.800542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.800700 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.800978 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.800979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.800979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.801031 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.801208 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.801209 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.801240 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.801288 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.801353 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 08:51:26.801424 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.794875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43342 (local address 127.15.208.61:34855)
0619 08:51:26.794913 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.794915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.794922 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:51:26.794953 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.794954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.794965 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.795013 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.795333 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.795898 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.796818 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.797043 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.797095 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.799233 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.799234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.799234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.799240 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.800446 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.800864 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.800865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.800865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.800918 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.801070 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.801071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.801071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.801161 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.801205 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.801303 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.801367 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:51:26.801815 23024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.60@KRBTEST.COM for oryx/127.15.208.60@KRBTEST.COM
I20260619 08:51:26.807433 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.802056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:35615 (local address 127.0.0.1:54938)
0619 08:51:26.802100 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.802110 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.802145 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.802244 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.802245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.802346 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.802371 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.802372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.802914 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.802914 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.803379 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.803381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.803698 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.803699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.803735 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.804136 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.804145 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.805271 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.806586 (+ 1315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.806587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.806587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.806741 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.806903 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.806903 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.806904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.806955 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.807168 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.807169 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.807204 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.807275 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.807351 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 08:51:26.807512 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.802100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.15.208.60:35615)
0619 08:51:26.802157 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.802158 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.802167 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:26.802184 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.802185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.802192 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.802240 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.802415 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.802852 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.803445 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.803639 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.803687 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.805341 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.805342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.805342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.805348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.806505 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.806804 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.806805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.806805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.806850 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.806993 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.806993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.806994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.807090 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.807134 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.807292 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.807393 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":33}
I20260619 08:51:26.808919 23024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Bootstrap starting.
I20260619 08:51:26.809767 23024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:26.810051 23024 log.cc:824] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:26.810734 23024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: No bootstrap required, opened a new log
I20260619 08:51:26.812237 23024 raft_consensus.cc:348] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } }
I20260619 08:51:26.812405 23024 raft_consensus.cc:374] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:26.812443 23024 raft_consensus.cc:729] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88ac38d150b425c9c1ddd9fe524fa12, State: Initialized, Role: FOLLOWER
I20260619 08:51:26.812599 23024 consensus_queue.cc:260] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } }
I20260619 08:51:26.812917 23024 sys_catalog.cc:564] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:26.812904 23029 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } } }
I20260619 08:51:26.813010 23029 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:51:26.814680 23040 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:51:26.814734 23040 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:51:26.816871 22956 master_runner.cc:428] Detected that this master a88ac38d150b425c9c1ddd9fe524fa12 is joining an existing cluster
I20260619 08:51:26.816916 22956 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.208.60:35615
I20260619 08:51:26.826475 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.817950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43352 (local address 127.15.208.61:34855)
0619 08:51:26.818074 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.818076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.818406 (+ 330us) server_negotiation.cc:407] Connection header received
0619 08:51:26.818465 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.818465 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.818476 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.818534 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.819043 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.819779 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.821069 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.821362 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.821405 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.822639 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.822640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.822640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.822646 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.824409 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.825285 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.825287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.825287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.825372 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.825648 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.825649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.825649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.825819 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.825885 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.826203 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.826344 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260619 08:51:26.826862 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.817974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34855 (local address 127.0.0.1:43352)
0619 08:51:26.818352 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.818362 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.818416 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.818588 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.818589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.818880 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.818948 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.818949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.819875 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.819875 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.820954 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.820957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.821602 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.821603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.821667 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.822081 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.822086 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.822543 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.824820 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.824821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.824821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.825164 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.825468 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.825468 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.825469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.825556 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.825963 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.825964 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.826047 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.826607 (+ 560us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.826750 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":49,"spinlock_wait_cycles":9600,"thread_start_us":111,"threads_started":1}
I20260619 08:51:26.849110 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.818269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.15.208.60:35615)
0619 08:51:26.818306 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.818308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.818542 (+ 234us) server_negotiation.cc:407] Connection header received
0619 08:51:26.818624 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.818625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.818632 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.818658 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.819070 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.819765 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.821134 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.821424 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.821480 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.846265 (+ 24785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.846266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.846266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.846273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.847959 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.848311 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.848312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.848312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.848364 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.848545 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.848546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.848546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.848676 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.848760 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.848921 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.849014 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:51:26.849131 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.818015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51502 (local address 127.15.208.62:46545)
0619 08:51:26.818067 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.818069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.818246 (+ 177us) server_negotiation.cc:407] Connection header received
0619 08:51:26.818360 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.818361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.818376 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.818431 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.818979 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.819778 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.821051 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.821429 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.821511 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.845976 (+ 24465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.845978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.845979 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.845988 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.847626 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.848081 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.848082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.848083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.848144 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.848370 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.848371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.848372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.848539 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.848608 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.848901 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.848987 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:51:26.849267 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 22366
I20260619 08:51:26.849963 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.817968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:51502)
0619 08:51:26.818170 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.818183 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.818273 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.818479 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.818480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.818809 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.818886 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.818887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.819863 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.819864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.820956 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.820959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.821585 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.821585 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.821639 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.845385 (+ 23746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.845389 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.845852 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.847749 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.847750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.847751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.847994 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.848210 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.848211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.848211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.848276 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.848680 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.848681 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.848751 (+ 70us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.849255 (+ 504us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.849786 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":102,"thread_start_us":138,"threads_started":1}
I20260619 08:51:26.850265 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.818248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:35615 (local address 127.0.0.1:54948)
0619 08:51:26.818496 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.818505 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.818562 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.818708 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.818708 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.818930 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.818991 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.818992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.819863 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.819864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.820956 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.820959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.821572 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.821573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.821631 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.845228 (+ 23597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.845235 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.845852 (+ 617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.848037 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.848038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.848039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.848244 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.848427 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.848428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.848428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.848476 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.848793 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.848794 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.848828 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.849621 (+ 793us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.849878 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":275,"thread_start_us":57,"threads_started":1}
I20260619 08:51:26.856493 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:36031
--local_ip_for_outbound_sockets=127.15.208.1
--tserver_master_addrs=127.15.208.62:46545
--webserver_port=46483
--webserver_interface=127.15.208.1
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab
--principal=oryx/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.208.62:46545,127.15.208.61:34855
--tserver_master_addrs=127.15.208.62:46545,127.15.208.61:34855,127.15.208.60:35615 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859085, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.62@KRBTEST.COM for oryx/127.15.208.60@KRBTEST.COM
I20260619 08:51:26.862942 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.855830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.15.208.60:35615)
0619 08:51:26.855867 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.855869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.855876 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:51:26.855906 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.855907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.855917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.855962 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.856344 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.857079 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.858098 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.858401 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.858477 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.860348 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.860350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.860350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.860358 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.861819 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.862181 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.862182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.862182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.862236 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.862435 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.862436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.862437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.862547 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.862622 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.862771 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.862846 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:51:26.862946 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.855637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:35615 (local address 127.0.0.1:54962)
0619 08:51:26.855699 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.855711 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.855757 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.856011 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.856012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.856203 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.856260 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.856262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.857165 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.857166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.858000 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.858003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.858483 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.858484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.858559 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.858958 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.858963 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.860258 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.861921 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.861922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.861923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.862097 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.862307 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.862308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.862309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.862372 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.862615 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.862616 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.862661 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.862746 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.862847 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 08:51:26.863446 22304 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.208.60:35615: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12"
member_type: NON_VOTER
last_known_addr {
host: "127.15.208.60"
port: 35615
}
attrs {
promote: true
}
}
dest_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2"
cas_config_opid_index: 5
I20260619 08:51:26.863659 22304 consensus_queue.cc:237] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } }
I20260619 08:51:26.863957 22863 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a88ac38d150b425c9c1ddd9fe524fa12 (127.15.208.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.864228 22998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:26.865074 22863 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.208.60:35615
I20260619 08:51:26.865181 22863 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.865262 22863 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:26.865648 22998 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 FOLLOWER]: Refusing update from remote peer da6a78a0b5bb473aa819bb6ce926b5b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 08:51:26.865942 22956 master.cc:537] Master@127.15.208.60:35615 shutting down...
I20260619 08:51:26.865917 22864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 08:51:26.866705 22291 consensus_peers.cc:487] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 -> Peer a88ac38d150b425c9c1ddd9fe524fa12 (127.15.208.60:35615): Couldn't send request to peer a88ac38d150b425c9c1ddd9fe524fa12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:51:26.869190 22956 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:26.869273 22956 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:26.869315 22956 tablet_replica.cc:330] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: stopping tablet replica
I20260619 08:51:26.870606 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.865448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34855 (local address 127.0.0.1:43366)
0619 08:51:26.867124 (+ 1676us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.867135 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.867185 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.867429 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.867430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.867431 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:26.867470 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.867471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.868302 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.868303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.869620 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.869623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.870329 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.870330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.870385 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.870408 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.870457 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.870493 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1658}
I20260619 08:51:26.870851 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.865568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43366 (local address 127.15.208.61:34855)
0619 08:51:26.865608 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.865610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.867240 (+ 1630us) server_negotiation.cc:407] Connection header received
0619 08:51:26.867270 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.867271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.867296 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.867352 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:26.867525 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.868233 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.869720 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.870228 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.870331 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.870400 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.870750 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.870790 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:51:26.871085 22912 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 LEARNER]: Refusing update from remote peer da6a78a0b5bb473aa819bb6ce926b5b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 08:51:26.871281 22864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 08:51:26.871847 22912 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a88ac38d150b425c9c1ddd9fe524fa12 (127.15.208.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.872819 22941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: SysCatalogTable state changed. Reason: New leader da6a78a0b5bb473aa819bb6ce926b5b2. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.872902 22941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: This master's current role is: LEARNER
I20260619 08:51:26.872938 22941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: NON_VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: true } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.872968 22864 raft_consensus.cc:1050] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: attempting to promote NON_VOTER 0806610f714f4ed295d35650ab23e71d to VOTER
I20260619 08:51:26.872978 22941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: This master's current role is: LEARNER
I20260619 08:51:26.873175 22864 consensus_queue.cc:237] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } }
I20260619 08:51:26.873873 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.873724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:35615 (local address 127.0.0.1:54964)
0619 08:51:26.873799 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.873840 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:35615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 08:51:26.874061 22291 consensus_peers.cc:487] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 -> Peer a88ac38d150b425c9c1ddd9fe524fa12 (127.15.208.60:35615): Couldn't send request to peer a88ac38d150b425c9c1ddd9fe524fa12. Status: Network error: Client connection negotiation failed: client connection to 127.15.208.60:35615: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 08:51:26.874351 22912 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 LEARNER]: Refusing update from remote peer da6a78a0b5bb473aa819bb6ce926b5b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 08:51:26.874523 23052 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 08:51:26.875129 22863 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0806610f714f4ed295d35650ab23e71d (127.15.208.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.875877 22863 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.875958 22863 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:26.875936 22864 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.875996 22864 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:26.876156 22912 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0806610f714f4ed295d35650ab23e71d (127.15.208.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.876909 22941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.876996 22941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:51:26.885277 22956 master.cc:552] Master@127.15.208.60:35615 shutdown complete.
I20260619 08:51:26.885341 22956 master_runner.cc:305] Clearing existing system tablet
I20260619 08:51:26.885471 22956 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:26.885533 22956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:26.886253 22956 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:26.886765 23054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.886926 22956 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:26.886984 22956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.887014 22956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal
uuid: "a88ac38d150b425c9c1ddd9fe524fa12"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "19a62e40cdc3171f4cbea3629c51a834"
server_key_iv: "f6628f78e83d00a58068faf3f3d90dad"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.887637 22956 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:26.888973 22956 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 08:51:26.889048 22956 log.cc:1196] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 08:51:26.889254 22956 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Deleting consensus metadata
I20260619 08:51:26.889394 22956 master_runner.cc:315] Copying system tablet from 127.15.208.62:46545
I20260619 08:51:26.890036 22956 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: tablet copy: Beginning tablet copy session from remote peer at address 127.15.208.62:46545
I20260619 08:51:26.896207 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.890394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51506 (local address 127.15.208.62:46545)
0619 08:51:26.890442 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:26.890444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:26.890580 (+ 136us) server_negotiation.cc:407] Connection header received
0619 08:51:26.890658 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:26.890660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:26.890670 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:26.890716 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:26.891016 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.891609 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.892311 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.892514 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.892593 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.893636 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:26.893638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:26.893638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:26.893646 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:26.895082 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.895436 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.895438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.895438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.895496 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:26.895685 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:26.895686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:26.895687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:26.895803 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:26.895884 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:26.896034 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 08:51:26.896113 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:51:26.896548 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:26.890338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:51506)
0619 08:51:26.890489 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:26.890508 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:26.890593 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:26.890728 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:26.890729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:26.890902 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:26.890937 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.890939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.891690 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.891691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.892219 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:26.892221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:26.892591 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:26.892592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:26.892646 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:26.893217 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:26.893222 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:26.893535 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:26.895185 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.895186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.895186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.895363 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.895571 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:26.895572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:26.895572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:26.895634 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:26.895875 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:26.895876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:26.895920 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 08:51:26.896010 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:51:26.896185 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260619 08:51:26.896936 22324 tablet_copy_service.cc:140] P da6a78a0b5bb473aa819bb6ce926b5b2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a88ac38d150b425c9c1ddd9fe524fa12 ({username='oryx', principal='oryx/127.15.208.60@KRBTEST.COM'} at 127.0.0.1:51506)
I20260619 08:51:26.897018 22324 tablet_copy_service.cc:161] P da6a78a0b5bb473aa819bb6ce926b5b2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a88ac38d150b425c9c1ddd9fe524fa12 at {username='oryx', principal='oryx/127.15.208.60@KRBTEST.COM'} at 127.0.0.1:51506: session id = a88ac38d150b425c9c1ddd9fe524fa12-00000000000000000000000000000000
I20260619 08:51:26.897569 22324 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 08:51:26.898018 22956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 08:51:26.899293 22956 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: tablet copy: Starting download of 0 data blocks...
I20260619 08:51:26.899446 22956 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: tablet copy: Starting download of 1 WAL segments...
I20260619 08:51:26.900110 22956 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 08:51:26.900856 22324 tablet_copy_service.cc:342] P da6a78a0b5bb473aa819bb6ce926b5b2: Request end of tablet copy session a88ac38d150b425c9c1ddd9fe524fa12-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.208.60@KRBTEST.COM'} at 127.0.0.1:51506
I20260619 08:51:26.900911 22324 tablet_copy_service.cc:433] P da6a78a0b5bb473aa819bb6ce926b5b2: ending tablet copy session a88ac38d150b425c9c1ddd9fe524fa12-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a88ac38d150b425c9c1ddd9fe524fa12
I20260619 08:51:26.903045 22956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:26.904208 23061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:26.905783 22956 server_base.cc:1034] running on GCE node
W20260619 08:51:26.905817 23064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.908870 23062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:26.909128 22956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:26.909350 22956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:26.910478 22956 hybrid_clock.cc:648] HybridClock initialized: now 1781859086910472 us; error 27 us; skew 500 ppm
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:26.911937 22956 init.cc:377] Logged in from keytab as oryx/127.15.208.60@KRBTEST.COM (short username oryx)
I20260619 08:51:26.912221 22956 webserver.cc:460] Webserver started at http://127.15.208.60:32867/ using document root <none> and password file <none>
I20260619 08:51:26.912338 22956 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:26.912372 22956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:26.913129 22956 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:26.913740 23071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.913932 22956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.913978 22956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal
uuid: "a88ac38d150b425c9c1ddd9fe524fa12"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "19a62e40cdc3171f4cbea3629c51a834"
server_key_iv: "f6628f78e83d00a58068faf3f3d90dad"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.914038 22956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:26.931445 22956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:26.933260 22956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:26.936684 22956 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.60:35615
I20260619 08:51:26.937088 22956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
I20260619 08:51:26.937387 23123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.60:35615 every 8 connection(s)
I20260619 08:51:26.937955 23124 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:51:26.938390 23124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Bootstrap starting.
I20260619 08:51:26.940778 23124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:51:26.940989 23124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Bootstrap complete.
I20260619 08:51:26.941198 23124 raft_consensus.cc:348] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } }
I20260619 08:51:26.941287 23124 raft_consensus.cc:729] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a88ac38d150b425c9c1ddd9fe524fa12, State: Initialized, Role: LEARNER
I20260619 08:51:26.941331 23124 consensus_queue.cc:260] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } }
I20260619 08:51:26.941497 23124 sys_catalog.cc:564] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:26.942752 23127 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:26.942842 23127 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: This master's current role is: LEARNER
I20260619 08:51:26.943573 23138 catalog_manager.cc:1259] Loaded cluster ID: 0defa86d538b4ccc9c8e6e52a435bad3
I20260619 08:51:26.943670 23138 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: loading cluster ID for follower catalog manager: success
I20260619 08:51:26.944514 23138 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: acquiring CA information for follower catalog manager: success
I20260619 08:51:26.944864 23138 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 08:51:26.978621 23049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:26.978907 23049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:26.978943 23049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:26.980696 23049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:26.980782 23049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:26.980822 23049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:26.983034 23049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab
--principal=oryx/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:36031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=46483
--tserver_master_addrs=127.15.208.62:46545,127.15.208.61:34855,127.15.208.60:35615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:26.983312 23049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:26.983566 23049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:26.988966 23146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:26.989085 23049 server_base.cc:1034] running on GCE node
W20260619 08:51:26.988972 23144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:26.989231 23143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:26.989429 23049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:26.989704 23049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:26.990880 23049 hybrid_clock.cc:648] HybridClock initialized: now 1781859086990867 us; error 31 us; skew 500 ppm
Jun 19 08:51:26 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:26.993475 23049 init.cc:377] Logged in from keytab as oryx/127.15.208.1@KRBTEST.COM (short username oryx)
I20260619 08:51:26.993983 23049 webserver.cc:460] Webserver started at http://127.15.208.1:46483/ using document root <none> and password file <none>
I20260619 08:51:26.994220 23049 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:26.994293 23049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:26.995849 23049 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:26.996421 23153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:26.996644 23049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:51:26.996763 23049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "851148a914324cc5ba28a76da4424c45"
format_stamp: "Formatted at 2026-06-19 08:51:25 on dist-test-slave-p0h9"
server_key: "347b57c2b826c1f1c1e00a6f896072dc"
server_key_iv: "ea99a3b171b6fc5d5d1365e513eaf437"
server_key_version: "encryptionkey@0"
I20260619 08:51:26.997121 23049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:27.009831 23049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:27.012038 23049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:27.012195 23049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:27.012426 23049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:27.012832 23049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:27.012883 23049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:27.012925 23049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:27.012952 23049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:27.019232 23049 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:36031
I20260619 08:51:27.019294 23266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:36031 every 8 connection(s)
I20260619 08:51:27.019652 23049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:27.023834 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 23049
I20260619 08:51:27.023983 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 22500
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.61@KRBTEST.COM
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.60@KRBTEST.COM
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859086, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:27.030808 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:46255
--local_ip_for_outbound_sockets=127.15.208.2
--tserver_master_addrs=127.15.208.62:46545
--webserver_port=45761
--webserver_interface=127.15.208.2
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab
--principal=oryx/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.208.62:46545,127.15.208.61:34855
--tserver_master_addrs=127.15.208.62:46545,127.15.208.61:34855,127.15.208.60:35615 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:51:27.032557 23272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.020276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:35615 (local address 127.15.208.1:39381)
0619 08:51:27.020624 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.020753 (+ 129us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.021355 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.021653 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.021658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.022125 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:27.022325 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.022330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.023508 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.023509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.024933 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.024936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.025395 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.025396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.025484 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.026156 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:27.026160 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:27.028454 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:27.030823 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.030830 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.030832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.031149 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.031351 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.031352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.031353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.031476 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.031781 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:27.031786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:27.031954 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.032088 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.032287 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260619 08:51:27.032876 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.020358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:34681 (local address 127.15.208.61:34855)
0619 08:51:27.020535 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.020537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.020881 (+ 344us) server_negotiation.cc:407] Connection header received
0619 08:51:27.021455 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.021456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.021467 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.021578 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.022432 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.023183 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.024915 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.025227 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.025269 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.029347 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.029348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.029349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.029356 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.031036 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.031980 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.031981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.031981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.032038 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.032283 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.032283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.032284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.032409 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.032459 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.032653 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.032791 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260619 08:51:27.033128 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.020356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:39381 (local address 127.15.208.60:35615)
0619 08:51:27.020459 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.020460 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.020845 (+ 385us) server_negotiation.cc:407] Connection header received
0619 08:51:27.021457 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.021458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.021471 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.021540 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.022428 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.023425 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.025020 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.025290 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.025346 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.028554 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.028556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.028556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.028564 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.030736 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.031231 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.031232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.031233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.031290 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.031538 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.031538 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.031539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.031699 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.031756 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.032868 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.032964 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":53,"threads_started":1}
I20260619 08:51:27.033133 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.020410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:58761 (local address 127.15.208.62:46545)
0619 08:51:27.020504 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.020507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.021610 (+ 1103us) server_negotiation.cc:407] Connection header received
0619 08:51:27.021717 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.021718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.021728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.021816 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.022416 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.022911 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.025767 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.026074 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.026140 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.029042 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.029043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.029044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.029058 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.031447 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.031980 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.031981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.031981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.032043 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.032271 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.032272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.032272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.032397 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.032453 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.032928 (+ 475us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.033022 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 08:51:27.033250 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.020276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34855 (local address 127.15.208.1:34681)
0619 08:51:27.020773 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.020783 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.021370 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.021654 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.021658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.022181 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:27.022325 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.022330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.023274 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.023275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.024824 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.024827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.025349 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.025349 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.025484 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.026083 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:27.026091 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:27.029262 (+ 3171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:27.031559 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.031560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.031560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.031889 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.032116 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.032117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.032117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.032209 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.032489 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:27.032489 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:27.032539 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.032981 (+ 442us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.033092 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":7,"thread_start_us":75,"threads_started":1}
I20260619 08:51:27.033478 23267 heartbeater.cc:347] Connected to a master server at 127.15.208.60:35615
I20260619 08:51:27.033565 23269 heartbeater.cc:347] Connected to a master server at 127.15.208.61:34855
I20260619 08:51:27.033629 23269 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:27.033785 23269 heartbeater.cc:511] Master 127.15.208.61:34855 requested a full tablet report, sending...
I20260619 08:51:27.033582 23267 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:27.034142 23267 heartbeater.cc:511] Master 127.15.208.60:35615 requested a full tablet report, sending...
I20260619 08:51:27.034276 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.020276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.1:58761)
0619 08:51:27.021528 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.021536 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.021633 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.021834 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.021835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.022131 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:27.022323 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.022327 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.023014 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.023015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.025689 (+ 2674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.025691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.026211 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.026212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.026289 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.026648 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:27.026652 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:27.028955 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:27.031541 (+ 2586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.031541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.031542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.031890 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.032116 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.032117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.032117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.032194 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.032488 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:27.032489 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:27.032532 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.033875 (+ 1343us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.033994 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1228,"mutex_wait_us":268,"thread_start_us":62,"threads_started":1}
I20260619 08:51:27.034297 22902 ts_manager.cc:194] Registered new tserver with Master: 851148a914324cc5ba28a76da4424c45 (127.15.208.1:36031)
I20260619 08:51:27.034727 23088 ts_manager.cc:194] Registered new tserver with Master: 851148a914324cc5ba28a76da4424c45 (127.15.208.1:36031)
I20260619 08:51:27.034822 23268 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46545
I20260619 08:51:27.034868 23268 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:27.034933 23268 heartbeater.cc:511] Master 127.15.208.62:46545 requested a full tablet report, sending...
I20260619 08:51:27.035256 22304 ts_manager.cc:194] Re-registered known tserver with Master: 851148a914324cc5ba28a76da4424c45 (127.15.208.1:36031)
I20260619 08:51:27.035646 22304 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:58761
W20260619 08:51:27.113619 23275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:27.113842 23275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:27.113880 23275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:27.115120 23275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:27.115192 23275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:27.115239 23275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:27.116770 23275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab
--principal=oryx/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:46255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=45761
--tserver_master_addrs=127.15.208.62:46545,127.15.208.61:34855,127.15.208.60:35615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:27.117040 23275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:27.117264 23275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:27.122092 23280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:27.122083 23281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:27.122202 23275 server_base.cc:1034] running on GCE node
W20260619 08:51:27.122063 23283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:27.122470 23275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:27.122740 23275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:27.123903 23275 hybrid_clock.cc:648] HybridClock initialized: now 1781859087123885 us; error 39 us; skew 500 ppm
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859087, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:27.126390 23275 init.cc:377] Logged in from keytab as oryx/127.15.208.2@KRBTEST.COM (short username oryx)
I20260619 08:51:27.126847 23275 webserver.cc:460] Webserver started at http://127.15.208.2:45761/ using document root <none> and password file <none>
I20260619 08:51:27.127074 23275 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:27.127152 23275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:27.128453 23275 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:27.129026 23290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:27.129189 23275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:27.129281 23275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "bc40bdaed75848d2b8a9d58ec180887a"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "4a4922fa78b37548196bdcf1b0cbb7a5"
server_key_iv: "72e4b09c0e73fd382e69eed191c629d1"
server_key_version: "encryptionkey@0"
I20260619 08:51:27.129611 23275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:27.148219 23275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:27.150422 23275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:27.150599 23275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:27.150835 23275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:27.151142 23275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:27.151192 23275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:27.151234 23275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:27.151260 23275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:27.157311 23275 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:46255
I20260619 08:51:27.157349 23403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:46255 every 8 connection(s)
I20260619 08:51:27.157737 23275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859087, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.61@KRBTEST.COM
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859087, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.60@KRBTEST.COM
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859087, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:27.166909 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 23275
I20260619 08:51:27.167078 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 22634
I20260619 08:51:27.168898 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.158754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:44663 (local address 127.15.208.61:34855)
0619 08:51:27.158797 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.158798 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.158941 (+ 143us) server_negotiation.cc:407] Connection header received
0619 08:51:27.159870 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.159871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.159884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.159917 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.160788 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.161538 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.162734 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.163030 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.163095 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.165301 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.165302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.165303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.165309 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.167296 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.167745 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.167746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.167746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.167809 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.168125 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.168127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.168127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.168274 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.168318 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.168695 (+ 377us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.168806 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:27.169029 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.158735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:45707 (local address 127.15.208.60:35615)
0619 08:51:27.158780 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.158782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.158875 (+ 93us) server_negotiation.cc:407] Connection header received
0619 08:51:27.159853 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.159854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.159868 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.159927 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.160761 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.161282 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.162299 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.162521 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.162605 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.165790 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.165791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.165791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.165798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.167489 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.168210 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.168211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.168211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.168258 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.168456 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.168457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.168457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.168585 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.168629 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.168856 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.168942 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:27.169456 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.158462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:35615 (local address 127.15.208.2:45707)
0619 08:51:27.158731 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.158778 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.159746 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.159997 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.160001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.160402 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:27.160658 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.160663 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.161393 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.161394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.162208 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.162210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.162615 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.162617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.162791 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.163427 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:27.163437 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:27.165665 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:27.167931 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.167932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.167932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.168128 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.168332 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.168332 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.168333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.168385 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.168677 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:27.168678 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:27.168712 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.169139 (+ 427us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.169249 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":29,"thread_start_us":98,"threads_started":1}
I20260619 08:51:27.169456 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.158462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34855 (local address 127.15.208.2:44663)
0619 08:51:27.158851 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.158862 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.159766 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.160010 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.160011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.160418 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:27.160658 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.160665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.161635 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.161636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.162605 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.162607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.163107 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.163108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.163187 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.163578 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:27.163582 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:27.165195 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:27.167376 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.167382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.167385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.167675 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.167878 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.167879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.167879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.168033 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.168366 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:27.168371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:27.168540 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.168992 (+ 452us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.169203 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":221,"thread_start_us":56,"threads_started":1}
I20260619 08:51:27.169895 23404 heartbeater.cc:347] Connected to a master server at 127.15.208.60:35615
I20260619 08:51:27.170014 23404 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:27.170190 23404 heartbeater.cc:511] Master 127.15.208.60:35615 requested a full tablet report, sending...
I20260619 08:51:27.170643 23088 ts_manager.cc:194] Registered new tserver with Master: bc40bdaed75848d2b8a9d58ec180887a (127.15.208.2:46255)
I20260619 08:51:27.172933 23406 heartbeater.cc:347] Connected to a master server at 127.15.208.61:34855
I20260619 08:51:27.173027 23406 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:27.173103 23406 heartbeater.cc:511] Master 127.15.208.61:34855 requested a full tablet report, sending...
I20260619 08:51:27.173379 22902 ts_manager.cc:194] Registered new tserver with Master: bc40bdaed75848d2b8a9d58ec180887a (127.15.208.2:46255)
I20260619 08:51:27.173751 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:45669
--local_ip_for_outbound_sockets=127.15.208.3
--tserver_master_addrs=127.15.208.62:46545
--webserver_port=33771
--webserver_interface=127.15.208.3
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab
--principal=oryx/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.208.62:46545,127.15.208.61:34855
--tserver_master_addrs=127.15.208.62:46545,127.15.208.61:34855,127.15.208.60:35615 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:51:27.174618 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.158462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.2:60535)
0619 08:51:27.158851 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.158862 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.159766 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.160003 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.160004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.160397 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:27.160658 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.160665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.161635 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.161636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.162604 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.162607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.163524 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.163525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.163588 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.163962 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:27.163966 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:27.166255 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:27.173529 (+ 7274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.173531 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.173531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.173724 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.173984 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.173984 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.173985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.174042 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.174366 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:27.174366 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:27.174402 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.174447 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.174531 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":71,"thread_start_us":180,"threads_started":1}
I20260619 08:51:27.174744 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.158527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:60535 (local address 127.15.208.62:46545)
0619 08:51:27.158634 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.158637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.158918 (+ 281us) server_negotiation.cc:407] Connection header received
0619 08:51:27.159871 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.159872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.159884 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.159917 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.160788 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.161534 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.163138 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.163401 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.163466 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.166792 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.166793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.166794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.166802 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.169382 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.173807 (+ 4425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.173809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.173809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.173909 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.174132 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.174134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.174134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.174300 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.174374 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.174483 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.174603 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:51:27.175067 23405 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46545
I20260619 08:51:27.175122 23405 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:27.175194 23405 heartbeater.cc:511] Master 127.15.208.62:46545 requested a full tablet report, sending...
I20260619 08:51:27.175392 22304 ts_manager.cc:194] Re-registered known tserver with Master: bc40bdaed75848d2b8a9d58ec180887a (127.15.208.2:46255)
I20260619 08:51:27.175717 22304 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:60535
W20260619 08:51:27.255729 23411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:27.255939 23411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:27.255961 23411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:27.257068 23411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:27.257104 23411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:27.257129 23411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:27.258605 23411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab
--principal=oryx/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:45669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=33771
--tserver_master_addrs=127.15.208.62:46545,127.15.208.61:34855,127.15.208.60:35615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:27.258796 23411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:27.258963 23411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:27.263960 23416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:27.263969 23419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:27.264001 23417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:27.264179 23411 server_base.cc:1034] running on GCE node
I20260619 08:51:27.264429 23411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:27.264698 23411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:27.265883 23411 hybrid_clock.cc:648] HybridClock initialized: now 1781859087265850 us; error 52 us; skew 500 ppm
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859087, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:27.268287 23411 init.cc:377] Logged in from keytab as oryx/127.15.208.3@KRBTEST.COM (short username oryx)
I20260619 08:51:27.268774 23411 webserver.cc:460] Webserver started at http://127.15.208.3:33771/ using document root <none> and password file <none>
I20260619 08:51:27.269022 23411 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:27.269094 23411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:27.270411 23411 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:27.270991 23426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:27.271176 23411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:27.271256 23411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "e8768e3237994432af741ad63b27d8d3"
format_stamp: "Formatted at 2026-06-19 08:51:26 on dist-test-slave-p0h9"
server_key: "307b7e9e29579682147e5f638064f8f6"
server_key_iv: "9a2320c379162d70babb349dddb43658"
server_key_version: "encryptionkey@0"
I20260619 08:51:27.271567 23411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:27.297267 23411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:27.299806 23411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:27.299923 23411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:27.300117 23411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:27.300462 23411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:27.300542 23411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:27.300590 23411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:27.300613 23411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:27.306591 23411 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:45669
I20260619 08:51:27.306650 23539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:45669 every 8 connection(s)
I20260619 08:51:27.307050 23411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:27.309324 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 23411
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859087, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.61@KRBTEST.COM
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859087, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.60@KRBTEST.COM
Jun 19 08:51:27 dist-test-slave-p0h9 krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859087, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:27.321033 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.308123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34855 (local address 127.15.208.3:42593)
0619 08:51:27.308234 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.308245 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.309217 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.309571 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.309574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.310114 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:27.310344 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.310349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.311281 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.311282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.312303 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.312305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.312873 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.312874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.312938 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.313676 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:27.313687 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:27.316461 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:27.319178 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.319184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.319187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.319514 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.319758 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.319759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.319759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.319893 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.320228 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:27.320234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:27.320421 (+ 187us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.320538 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.320765 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":191,"thread_start_us":62,"threads_started":1}
I20260619 08:51:27.321341 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.308370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:42593 (local address 127.15.208.61:34855)
0619 08:51:27.308438 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.308439 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.308446 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:51:27.309324 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.309326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.309339 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.309372 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.310453 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.311197 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.312399 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.312741 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.312818 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.316551 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.316553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.316553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.316560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.319106 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.319614 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.319615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.319615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.319679 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.319975 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.319976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.319977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.320121 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.320198 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.321170 (+ 972us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.321244 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:51:27.321755 23542 heartbeater.cc:347] Connected to a master server at 127.15.208.61:34855
I20260619 08:51:27.321861 23542 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:27.322064 23542 heartbeater.cc:511] Master 127.15.208.61:34855 requested a full tablet report, sending...
I20260619 08:51:27.322445 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.307792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:51199 (local address 127.15.208.60:35615)
0619 08:51:27.308240 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.308242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.308253 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:51:27.309325 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.309326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.309339 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.309372 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.310453 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.311209 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.312237 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.312546 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.312627 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.316304 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.316305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.316306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.316312 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.321040 (+ 4728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.321411 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.321412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.321412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.321465 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.321663 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.321664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.321664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.321780 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.322207 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.322247 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.322325 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420}
I20260619 08:51:27.322499 22902 ts_manager.cc:194] Registered new tserver with Master: e8768e3237994432af741ad63b27d8d3 (127.15.208.3:45669)
I20260619 08:51:27.322706 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.307809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:35615 (local address 127.15.208.3:51199)
0619 08:51:27.307978 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.308024 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.309212 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.309571 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.309574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.310078 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:27.310344 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.310349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.311297 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.311298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.312148 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.312150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.312713 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.312714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.312901 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.313856 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:27.313861 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:27.316242 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:27.321122 (+ 4880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.321123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.321123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.321341 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.321528 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.321529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.321529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.321589 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.321963 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:27.321964 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:27.322022 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.322086 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.322606 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":14,"thread_start_us":114,"threads_started":1}
I20260619 08:51:27.322723 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.312547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43376 (local address 127.15.208.61:34855)
0619 08:51:27.312884 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.312886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.313449 (+ 563us) server_negotiation.cc:407] Connection header received
0619 08:51:27.313868 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.313869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.313881 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.313920 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.314875 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.315639 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.316607 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.316946 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.317028 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.318810 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.318811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.318811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.318817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.321146 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.321515 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.321516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.321517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.321569 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.321772 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.321773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.321773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.321892 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.321981 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.322550 (+ 569us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.322635 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":70,"threads_started":1}
I20260619 08:51:27.323175 23540 heartbeater.cc:347] Connected to a master server at 127.15.208.60:35615
I20260619 08:51:27.323233 23540 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:27.323287 23540 heartbeater.cc:511] Master 127.15.208.60:35615 requested a full tablet report, sending...
I20260619 08:51:27.323745 23088 ts_manager.cc:194] Registered new tserver with Master: e8768e3237994432af741ad63b27d8d3 (127.15.208.3:45669)
I20260619 08:51:27.325299 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.307809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.3:55447)
0619 08:51:27.307984 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.308024 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.309197 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.309568 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.309574 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.310118 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:27.310343 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.310348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.311064 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.311065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.312308 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.312310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.312919 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.312920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.312990 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.316599 (+ 3609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:27.316604 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:27.318069 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:27.322087 (+ 4018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.322088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.322089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.322337 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.324495 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:27.324496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:27.324496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:27.324578 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:27.324931 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:27.324932 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:27.324993 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.325061 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.325162 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":53,"thread_start_us":91,"threads_started":1}
I20260619 08:51:27.325426 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.307811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:55447 (local address 127.15.208.62:46545)
0619 08:51:27.307862 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.307865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.308138 (+ 273us) server_negotiation.cc:407] Connection header received
0619 08:51:27.309336 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.309337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.309351 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.309384 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.310441 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.310949 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.312407 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.312796 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.312833 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.318178 (+ 5345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.318179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.318180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.318187 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.320116 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.324344 (+ 4228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.324345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.324346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.324414 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.324658 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.324659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.324659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.324846 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.324920 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.325131 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.325309 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:51:27.325702 23541 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46545
I20260619 08:51:27.325754 23541 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:27.325830 23541 heartbeater.cc:511] Master 127.15.208.62:46545 requested a full tablet report, sending...
I20260619 08:51:27.326061 22304 ts_manager.cc:194] Re-registered known tserver with Master: e8768e3237994432af741ad63b27d8d3 (127.15.208.3:45669)
I20260619 08:51:27.326411 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.313580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54972 (local address 127.15.208.60:35615)
0619 08:51:27.314948 (+ 1368us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.314949 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.315109 (+ 160us) server_negotiation.cc:407] Connection header received
0619 08:51:27.315129 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.315129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.315140 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.315190 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.317264 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.318059 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.319097 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.319405 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.319461 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.320865 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.320866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.320866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.320873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.325015 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.325545 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.325546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.325546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.325606 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.325838 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.325839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.325839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.326011 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.326179 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.326223 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.326307 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1350,"thread_start_us":68,"threads_started":1}
I20260619 08:51:27.326565 22304 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:55447
I20260619 08:51:27.327431 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.311460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51518 (local address 127.15.208.62:46545)
0619 08:51:27.313114 (+ 1654us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.313116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.313126 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:27.313149 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.313150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.313164 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.313259 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.314189 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.314963 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.316035 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.319299 (+ 3264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.321009 (+ 1710us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.324021 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.324023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.324023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.324029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.326138 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.326545 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.326546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.326547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.326653 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.326869 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.326870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.326871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.326975 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.327069 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.327249 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.327331 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1626,"thread_start_us":80,"threads_started":1}
I20260619 08:51:27.328894 22304 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:27.330423 22304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:27.338685 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.334224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:36031 (local address 127.0.0.1:46412)
0619 08:51:27.334488 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.334496 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.334546 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.334986 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.334987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.334988 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:27.335025 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.335026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.336328 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.336329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.337213 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.337216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.338430 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.338431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.338492 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.338497 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.338547 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.338603 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":49,"thread_start_us":68,"threads_started":1}
I20260619 08:51:27.338810 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.334187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:45669 (local address 127.0.0.1:38036)
0619 08:51:27.334233 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.334244 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.334282 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.334664 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.334664 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.334666 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:27.334711 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.334712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.336392 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.336393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.337573 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.337576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.338490 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.338492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.338561 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.338565 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.338621 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.338667 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":34}
I20260619 08:51:27.338919 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.334262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38036 (local address 127.15.208.3:45669)
0619 08:51:27.334506 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.334511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.334523 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:51:27.334562 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.334565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.334582 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.334662 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:27.334796 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.336334 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.337661 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.338397 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.338471 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.338542 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.338633 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.338669 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":106,"threads_started":1}
I20260619 08:51:27.339046 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.334430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.15.208.1:36031)
0619 08:51:27.334776 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.334781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.334791 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:27.334829 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.334831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.334847 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.334917 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:27.335104 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.336242 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.337327 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.337978 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.338049 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.338389 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.338904 (+ 515us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.338951 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":76,"threads_started":1}
I20260619 08:51:27.339206 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.334215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:46255 (local address 127.0.0.1:39274)
0619 08:51:27.334367 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.334377 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.334412 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.335282 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.335284 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.335286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:27.335321 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.335322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.336313 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.336314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.337832 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.337834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.338712 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.338712 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.338780 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.338783 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.339100 (+ 317us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.339144 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":97,"thread_start_us":82,"threads_started":1}
I20260619 08:51:27.339470 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.334433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.15.208.2:46255)
0619 08:51:27.335131 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.335134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.335141 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:51:27.335181 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.335183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.335194 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.335273 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:27.335371 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.336229 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.337916 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.338407 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.338500 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.338575 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.339348 (+ 773us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.339396 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":76,"threads_started":1}
I20260619 08:51:27.339664 23474 tablet_service.cc:1467] Processing CreateTablet for tablet b5795f1043094edfaa40d9e9bf571754 (DEFAULT_TABLE table=test-table [id=fca9e5488d5e44c286a752de8bdf4e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:27.339681 23201 tablet_service.cc:1467] Processing CreateTablet for tablet b5795f1043094edfaa40d9e9bf571754 (DEFAULT_TABLE table=test-table [id=fca9e5488d5e44c286a752de8bdf4e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:27.340039 23201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5795f1043094edfaa40d9e9bf571754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:27.340052 23474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5795f1043094edfaa40d9e9bf571754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:27.340001 23338 tablet_service.cc:1467] Processing CreateTablet for tablet b5795f1043094edfaa40d9e9bf571754 (DEFAULT_TABLE table=test-table [id=fca9e5488d5e44c286a752de8bdf4e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:27.340276 23338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5795f1043094edfaa40d9e9bf571754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:27.343149 23566 tablet_bootstrap.cc:492] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: Bootstrap starting.
I20260619 08:51:27.343837 23568 tablet_bootstrap.cc:492] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: Bootstrap starting.
I20260619 08:51:27.343936 23566 tablet_bootstrap.cc:654] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:27.344223 23566 log.cc:824] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:27.344619 23568 tablet_bootstrap.cc:654] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:27.344910 23567 tablet_bootstrap.cc:492] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: Bootstrap starting.
I20260619 08:51:27.344982 23568 log.cc:824] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:27.345211 23566 tablet_bootstrap.cc:492] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: No bootstrap required, opened a new log
I20260619 08:51:27.345290 23566 ts_tablet_manager.cc:1397] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:27.345592 23567 tablet_bootstrap.cc:654] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:27.345682 23568 tablet_bootstrap.cc:492] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: No bootstrap required, opened a new log
I20260619 08:51:27.345746 23568 ts_tablet_manager.cc:1397] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:27.345906 23567 log.cc:824] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:27.346890 23567 tablet_bootstrap.cc:492] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: No bootstrap required, opened a new log
I20260619 08:51:27.346959 23567 ts_tablet_manager.cc:1397] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:27.347510 23566 raft_consensus.cc:348] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } } peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } }
I20260619 08:51:27.347680 23566 raft_consensus.cc:374] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:27.347714 23566 raft_consensus.cc:729] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 851148a914324cc5ba28a76da4424c45, State: Initialized, Role: FOLLOWER
I20260619 08:51:27.347842 23566 consensus_queue.cc:260] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } } peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } }
I20260619 08:51:27.347985 23568 raft_consensus.cc:348] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } } peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } }
I20260619 08:51:27.348120 23566 ts_tablet_manager.cc:1428] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:51:27.348167 23568 raft_consensus.cc:374] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:27.348217 23568 raft_consensus.cc:729] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc40bdaed75848d2b8a9d58ec180887a, State: Initialized, Role: FOLLOWER
I20260619 08:51:27.348240 23268 heartbeater.cc:503] Master 127.15.208.62:46545 was elected leader, sending a full tablet report...
I20260619 08:51:27.348325 23568 consensus_queue.cc:260] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } } peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } }
I20260619 08:51:27.348556 23568 ts_tablet_manager.cc:1428] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:51:27.349131 23405 heartbeater.cc:503] Master 127.15.208.62:46545 was elected leader, sending a full tablet report...
I20260619 08:51:27.350250 23567 raft_consensus.cc:348] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } } peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } }
I20260619 08:51:27.350425 23567 raft_consensus.cc:374] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:27.350461 23567 raft_consensus.cc:729] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8768e3237994432af741ad63b27d8d3, State: Initialized, Role: FOLLOWER
I20260619 08:51:27.350584 23567 consensus_queue.cc:260] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } } peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } }
I20260619 08:51:27.350776 23567 ts_tablet_manager.cc:1428] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 08:51:27.350848 23541 heartbeater.cc:503] Master 127.15.208.62:46545 was elected leader, sending a full tablet report...
W20260619 08:51:27.408529 23407 tablet.cc:2369] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:51:27.468757 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.466425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54980 (local address 127.15.208.60:35615)
0619 08:51:27.466493 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.466495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.466505 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:27.466554 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.466555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.466574 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.466624 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:27.466763 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.467247 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.468116 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.468495 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.468558 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.468614 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.468658 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.468684 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:51:27.468819 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.466256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:35615 (local address 127.0.0.1:54980)
0619 08:51:27.466402 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.466418 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.466487 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.466647 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.466648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.466650 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:27.466698 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.466699 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.467311 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.467311 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.468024 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.468025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.468568 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.468569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.468611 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.468614 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.468646 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.468686 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260619 08:51:27.469234 23096 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260619 08:51:27.470170 23575 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da6a78a0b5bb473aa819bb6ce926b5b2. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:27.470274 23575 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: This master's current role is: LEARNER
I20260619 08:51:27.507758 23572 raft_consensus.cc:482] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:27.507901 23572 raft_consensus.cc:504] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } } peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } }
I20260619 08:51:27.508488 23572 leader_election.cc:288] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8768e3237994432af741ad63b27d8d3 (127.15.208.3:45669), bc40bdaed75848d2b8a9d58ec180887a (127.15.208.2:46255)
I20260619 08:51:27.511951 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.508875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:33975 (local address 127.15.208.3:45669)
0619 08:51:27.508917 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.508920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.508930 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:27.509023 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.509025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.509059 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.509114 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:27.509269 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.509736 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.511089 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.511615 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.511695 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.511704 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.511845 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.511883 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:51:27.511960 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.508814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:45669 (local address 127.15.208.1:33975)
0619 08:51:27.508867 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.508878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.508921 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.509129 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.509129 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.509131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:27.509190 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.509192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.509818 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.509819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.510995 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.510997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.511703 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.511704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.511760 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.511764 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.511813 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.511878 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:51:27.512140 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.508733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:46255 (local address 127.15.208.1:47555)
0619 08:51:27.508815 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:27.508828 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:27.508893 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:27.509088 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:27.509089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:27.509091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:27.509429 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.509431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.510124 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.510125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.511137 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:27.511139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.511880 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.511881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:27.511949 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.511951 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:27.512005 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:51:27.512052 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 08:51:27.512151 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.508849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:47555 (local address 127.15.208.2:46255)
0619 08:51:27.508893 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.508894 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.508906 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:51:27.508979 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.508980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.508990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.509065 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:27.509537 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.510005 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.511207 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.511781 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.511866 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.511873 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.512024 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.512071 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:51:27.512387 23494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5795f1043094edfaa40d9e9bf571754" candidate_uuid: "851148a914324cc5ba28a76da4424c45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8768e3237994432af741ad63b27d8d3" is_pre_election: true
I20260619 08:51:27.512418 23358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5795f1043094edfaa40d9e9bf571754" candidate_uuid: "851148a914324cc5ba28a76da4424c45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc40bdaed75848d2b8a9d58ec180887a" is_pre_election: true
I20260619 08:51:27.512557 23494 raft_consensus.cc:2454] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 851148a914324cc5ba28a76da4424c45 in term 0.
I20260619 08:51:27.512630 23358 raft_consensus.cc:2454] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 851148a914324cc5ba28a76da4424c45 in term 0.
I20260619 08:51:27.512857 23155 leader_election.cc:302] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 851148a914324cc5ba28a76da4424c45, e8768e3237994432af741ad63b27d8d3; no voters:
I20260619 08:51:27.513062 23572 raft_consensus.cc:2781] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:27.513187 23572 raft_consensus.cc:482] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:27.513212 23572 raft_consensus.cc:3037] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:27.514107 23572 raft_consensus.cc:504] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } } peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } }
I20260619 08:51:27.514268 23572 leader_election.cc:288] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [CANDIDATE]: Term 1 election: Requested vote from peers e8768e3237994432af741ad63b27d8d3 (127.15.208.3:45669), bc40bdaed75848d2b8a9d58ec180887a (127.15.208.2:46255)
I20260619 08:51:27.514377 23494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5795f1043094edfaa40d9e9bf571754" candidate_uuid: "851148a914324cc5ba28a76da4424c45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8768e3237994432af741ad63b27d8d3"
I20260619 08:51:27.514401 23358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5795f1043094edfaa40d9e9bf571754" candidate_uuid: "851148a914324cc5ba28a76da4424c45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc40bdaed75848d2b8a9d58ec180887a"
I20260619 08:51:27.514452 23494 raft_consensus.cc:3037] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:27.514497 23358 raft_consensus.cc:3037] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:27.515326 23494 raft_consensus.cc:2454] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 851148a914324cc5ba28a76da4424c45 in term 1.
I20260619 08:51:27.515386 23358 raft_consensus.cc:2454] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 851148a914324cc5ba28a76da4424c45 in term 1.
I20260619 08:51:27.515491 23155 leader_election.cc:302] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 851148a914324cc5ba28a76da4424c45, e8768e3237994432af741ad63b27d8d3; no voters:
I20260619 08:51:27.515599 23572 raft_consensus.cc:2781] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:27.515715 23572 raft_consensus.cc:686] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 1 LEADER]: Becoming Leader. State: Replica: 851148a914324cc5ba28a76da4424c45, State: Running, Role: LEADER
I20260619 08:51:27.515861 23572 consensus_queue.cc:237] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } } peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } }
I20260619 08:51:27.516669 22304 catalog_manager.cc:5495] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 851148a914324cc5ba28a76da4424c45 (127.15.208.1). New cstate: current_term: 1 leader_uuid: "851148a914324cc5ba28a76da4424c45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } health_report { overall_health: HEALTHY } } }
I20260619 08:51:27.517325 22864 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: NON_VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: true } } }
I20260619 08:51:27.517427 22864 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:27.518049 23582 raft_consensus.cc:1050] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: attempting to promote NON_VOTER a88ac38d150b425c9c1ddd9fe524fa12 to VOTER
I20260619 08:51:27.518283 23582 consensus_queue.cc:237] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } }
I20260619 08:51:27.519086 23096 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 LEARNER]: Refusing update from remote peer da6a78a0b5bb473aa819bb6ce926b5b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 08:51:27.519490 22864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 08:51:27.519575 22912 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 FOLLOWER]: Refusing update from remote peer da6a78a0b5bb473aa819bb6ce926b5b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 08:51:27.519757 23582 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 08:51:27.520242 23581 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a88ac38d150b425c9c1ddd9fe524fa12 (127.15.208.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:27.520529 22912 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a88ac38d150b425c9c1ddd9fe524fa12 (127.15.208.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:27.520933 23096 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a88ac38d150b425c9c1ddd9fe524fa12 (127.15.208.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
W20260619 08:51:27.521227 23270 tablet.cc:2369] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:51:27.521293 23581 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:27.521358 23581 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:27.521409 23581 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:27.521461 23581 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:27.521502 23581 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:27.521555 23581 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:27.521730 23585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:27.521850 23585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:51:27.522019 23575 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:27.522100 23575 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:51:27.524037 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.521670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46414 (local address 127.15.208.1:36031)
0619 08:51:27.521722 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.521724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.521731 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:51:27.521766 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.521767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.521780 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.521867 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:27.522167 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.522675 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.523426 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.523607 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.523661 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.523795 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:27.523886 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:27.523964 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.523976 (+ 12us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.523991 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:51:27.525723 23358 raft_consensus.cc:1261] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a [term 1 FOLLOWER]: Refusing update from remote peer 851148a914324cc5ba28a76da4424c45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:27.525723 23494 raft_consensus.cc:1261] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3 [term 1 FOLLOWER]: Refusing update from remote peer 851148a914324cc5ba28a76da4424c45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:27.526070 23579 consensus_queue.cc:1035] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:27.526238 23572 consensus_queue.cc:1035] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:27.528489 23586 mvcc.cc:205] Tried to move back new op lower bound from 7298494822503047168 to 7298494822465531904. Current Snapshot: MvccSnapshot[applied={T|T < 7298494822503047168}]
I20260619 08:51:27.528887 23587 mvcc.cc:205] Tried to move back new op lower bound from 7298494822503047168 to 7298494822465531904. Current Snapshot: MvccSnapshot[applied={T|T < 7298494822503047168}]
I20260619 08:51:27.530897 22303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:27.530987 22303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:27.532244 22303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da6a78a0b5bb473aa819bb6ce926b5b2: Sending DeleteTablet for 3 replicas of tablet b5795f1043094edfaa40d9e9bf571754
I20260619 08:51:27.532582 23474 tablet_service.cc:1514] Processing DeleteTablet for tablet b5795f1043094edfaa40d9e9bf571754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:27 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:38036
I20260619 08:51:27.532637 23201 tablet_service.cc:1514] Processing DeleteTablet for tablet b5795f1043094edfaa40d9e9bf571754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:27 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:46412
I20260619 08:51:27.532698 23338 tablet_service.cc:1514] Processing DeleteTablet for tablet b5795f1043094edfaa40d9e9bf571754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:27 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:39274
I20260619 08:51:27.532810 23595 tablet_replica.cc:330] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: stopping tablet replica
I20260619 08:51:27.532899 23595 raft_consensus.cc:2229] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:27.533004 23595 raft_consensus.cc:2258] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:27.533288 23597 tablet_replica.cc:330] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: stopping tablet replica
I20260619 08:51:27.533383 23597 raft_consensus.cc:2229] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:27.533480 23597 raft_consensus.cc:2258] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:27.533628 23596 tablet_replica.cc:330] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: stopping tablet replica
I20260619 08:51:27.533707 23596 raft_consensus.cc:2229] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:27.533787 23596 raft_consensus.cc:2258] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:27.534003 23597 ts_tablet_manager.cc:1905] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:27.534229 23596 ts_tablet_manager.cc:1905] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:27.534677 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 22272
I20260619 08:51:27.535900 23597 ts_tablet_manager.cc:1918] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:27.535974 23597 log.cc:1196] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/b5795f1043094edfaa40d9e9bf571754
I20260619 08:51:27.536258 23597 ts_tablet_manager.cc:1939] T b5795f1043094edfaa40d9e9bf571754 P bc40bdaed75848d2b8a9d58ec180887a: Deleting consensus metadata
I20260619 08:51:27.536546 23596 ts_tablet_manager.cc:1918] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:27.536677 23596 log.cc:1196] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/b5795f1043094edfaa40d9e9bf571754
I20260619 08:51:27.536989 23596 ts_tablet_manager.cc:1939] T b5795f1043094edfaa40d9e9bf571754 P 851148a914324cc5ba28a76da4424c45: Deleting consensus metadata
W20260619 08:51:27.541107 23292 connection.cc:535] server connection from 127.0.0.1:39274 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 08:51:27.541674 23157 connection.cc:535] server connection from 127.0.0.1:46412 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:51:27.541723 23595 ts_tablet_manager.cc:1905] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:27.543642 23595 ts_tablet_manager.cc:1918] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:27.543720 23595 log.cc:1196] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/b5795f1043094edfaa40d9e9bf571754
I20260619 08:51:27.544000 23595 ts_tablet_manager.cc:1939] T b5795f1043094edfaa40d9e9bf571754 P e8768e3237994432af741ad63b27d8d3: Deleting consensus metadata
W20260619 08:51:27.544405 23430 connection.cc:441] server connection from 127.0.0.1:38036 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38036 (request call id 1) could send its response
I20260619 08:51:27.552193 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.533690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54996 (local address 127.15.208.60:35615)
0619 08:51:27.533730 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.533732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.533741 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:27.533762 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.533763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.533773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.533830 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.544747 (+ 10917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.545545 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.546641 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.546963 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.547024 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.548870 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.548872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.548872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.548879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.550805 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.551189 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.551190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.551191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.551244 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.551437 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.551438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.551438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.551553 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.551641 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.552002 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.552094 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:51:27.552421 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.543107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43382 (local address 127.15.208.61:34855)
0619 08:51:27.543149 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.543151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.543358 (+ 207us) server_negotiation.cc:407] Connection header received
0619 08:51:27.543530 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.543531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.543545 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.543580 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.544286 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.545081 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.546665 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.546981 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.547022 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.549422 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.549423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.549423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.549431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.551307 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.551642 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.551643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.551644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.551692 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.551870 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.551871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.551871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.552007 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.552074 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.552266 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.552353 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:51:27.553061 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:27.543390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.15.208.60:35615)
0619 08:51:27.543448 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:27.543450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:27.544880 (+ 1430us) server_negotiation.cc:407] Connection header received
0619 08:51:27.544967 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:27.544968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:27.544978 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:27.545018 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:27.545967 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.546661 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.547610 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:27.547899 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:27.547963 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:27.549425 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:27.549426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:27.549426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:27.549433 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:27.551357 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.551866 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.551867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.551867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.551921 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:27.552130 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:27.552131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:27.552131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:27.552241 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:27.552312 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:27.552931 (+ 619us) server_negotiation.cc:299] Negotiation successful
0619 08:51:27.553006 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":35}
I20260619 08:51:28.529863 23605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:28.529592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.1:38031)
0619 08:51:28.529705 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:28.529793 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
W20260619 08:51:28.530119 23268 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:46545 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
I20260619 08:51:28.530808 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:28.530645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.2:49797)
0619 08:51:28.530729 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:28.530775 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":42,"threads_started":1}
W20260619 08:51:28.530997 23405 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:46545 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
I20260619 08:51:28.532166 23607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:28.531834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.3:35249)
0619 08:51:28.532055 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:28.532131 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":56,"threads_started":1}
W20260619 08:51:28.532339 23541 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:46545 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
I20260619 08:51:29.032656 23608 raft_consensus.cc:482] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da6a78a0b5bb473aa819bb6ce926b5b2)
I20260619 08:51:29.032800 23609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da6a78a0b5bb473aa819bb6ce926b5b2)
I20260619 08:51:29.032829 23608 raft_consensus.cc:504] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } }
I20260619 08:51:29.032915 23609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } }
I20260619 08:51:29.033113 23608 leader_election.cc:288] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da6a78a0b5bb473aa819bb6ce926b5b2 (127.15.208.62:46545), 0806610f714f4ed295d35650ab23e71d (127.15.208.61:34855)
I20260619 08:51:29.033454 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.033197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:52004)
0619 08:51:29.033364 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.033413 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":48,"threads_started":1}
W20260619 08:51:29.033684 23072 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
I20260619 08:51:29.033881 23609 leader_election.cc:288] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da6a78a0b5bb473aa819bb6ce926b5b2 (127.15.208.62:46545), a88ac38d150b425c9c1ddd9fe524fa12 (127.15.208.60:35615)
I20260619 08:51:29.034053 23614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.033440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:52020)
0619 08:51:29.033983 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.034021 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":41,"threads_started":1}
W20260619 08:51:29.034152 22886 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
I20260619 08:51:29.034425 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.034331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:52026)
0619 08:51:29.034371 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.034394 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 08:51:29.034509 23072 leader_election.cc:334] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da6a78a0b5bb473aa819bb6ce926b5b2 (127.15.208.62:46545): Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
I20260619 08:51:29.034825 23613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.034741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:52040)
0619 08:51:29.034773 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.034800 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260619 08:51:29.034893 22886 leader_election.cc:334] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da6a78a0b5bb473aa819bb6ce926b5b2 (127.15.208.62:46545): Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
I20260619 08:51:29.036643 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.033232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34855 (local address 127.0.0.1:43386)
0619 08:51:29.033495 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.033505 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:29.033587 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:29.033782 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:29.033783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:29.033786 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:29.033843 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.033845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.034815 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.034816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.035904 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.035907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.036476 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.036477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.036520 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.036523 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:29.036566 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 08:51:29.036598 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":54,"threads_started":1}
I20260619 08:51:29.036684 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.033318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.15.208.61:34855)
0619 08:51:29.033401 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:29.033404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:29.033563 (+ 159us) server_negotiation.cc:407] Connection header received
0619 08:51:29.033658 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:29.033659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:29.033669 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:29.033707 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:29.033941 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.034722 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.036000 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.036365 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.036467 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.036477 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:29.036585 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 08:51:29.036621 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":66,"thread_start_us":39,"threads_started":1}
I20260619 08:51:29.037026 22912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0806610f714f4ed295d35650ab23e71d" is_pre_election: true
I20260619 08:51:29.037145 22912 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88ac38d150b425c9c1ddd9fe524fa12 in term 1.
I20260619 08:51:29.037334 23614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.033607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:35615 (local address 127.0.0.1:55018)
0619 08:51:29.034328 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.034338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:29.034391 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:29.034592 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:29.034593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:29.034596 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:29.034636 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.034637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.035387 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.035388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.036345 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.036348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.037101 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.037102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.037172 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.037176 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:29.037221 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 08:51:29.037263 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":1,"thread_start_us":36,"threads_started":1}
I20260619 08:51:29.037319 23074 leader_election.cc:302] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0806610f714f4ed295d35650ab23e71d, a88ac38d150b425c9c1ddd9fe524fa12; no voters: da6a78a0b5bb473aa819bb6ce926b5b2
I20260619 08:51:29.037398 23616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.033770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.15.208.60:35615)
0619 08:51:29.033914 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:29.033916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:29.034444 (+ 528us) server_negotiation.cc:407] Connection header received
0619 08:51:29.034466 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:29.034467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:29.034477 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:29.034538 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:29.034719 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.035302 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.036424 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.037021 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.037082 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.037091 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:29.037247 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 08:51:29.037283 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260619 08:51:29.037496 23608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 08:51:29.037536 23608 raft_consensus.cc:482] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da6a78a0b5bb473aa819bb6ce926b5b2)
I20260619 08:51:29.037555 23608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:51:29.037592 23096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0806610f714f4ed295d35650ab23e71d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" is_pre_election: true
I20260619 08:51:29.038272 23608 raft_consensus.cc:504] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } }
I20260619 08:51:29.038431 23096 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0806610f714f4ed295d35650ab23e71d in current term 2: Already voted for candidate a88ac38d150b425c9c1ddd9fe524fa12 in this term.
I20260619 08:51:29.038437 23608 leader_election.cc:288] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [CANDIDATE]: Term 2 election: Requested vote from peers da6a78a0b5bb473aa819bb6ce926b5b2 (127.15.208.62:46545), 0806610f714f4ed295d35650ab23e71d (127.15.208.61:34855)
I20260619 08:51:29.038587 22912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0806610f714f4ed295d35650ab23e71d"
I20260619 08:51:29.038626 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.038509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:52052)
0619 08:51:29.038578 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.038606 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 08:51:29.038645 22889 leader_election.cc:302] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0806610f714f4ed295d35650ab23e71d; no voters: a88ac38d150b425c9c1ddd9fe524fa12, da6a78a0b5bb473aa819bb6ce926b5b2
I20260619 08:51:29.038664 22912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:51:29.038898 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.038810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:52068)
0619 08:51:29.038844 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.038872 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260619 08:51:29.038981 23072 leader_election.cc:334] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer da6a78a0b5bb473aa819bb6ce926b5b2 (127.15.208.62:46545): Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
I20260619 08:51:29.039454 22912 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88ac38d150b425c9c1ddd9fe524fa12 in term 2.
I20260619 08:51:29.039544 23609 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 08:51:29.039629 23074 leader_election.cc:302] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0806610f714f4ed295d35650ab23e71d, a88ac38d150b425c9c1ddd9fe524fa12; no voters: da6a78a0b5bb473aa819bb6ce926b5b2
I20260619 08:51:29.039728 23608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:51:29.039883 23608 raft_consensus.cc:686] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [term 2 LEADER]: Becoming Leader. State: Replica: a88ac38d150b425c9c1ddd9fe524fa12, State: Running, Role: LEADER
I20260619 08:51:29.039955 23608 consensus_queue.cc:237] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } }
I20260619 08:51:29.040288 23618 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a88ac38d150b425c9c1ddd9fe524fa12. Latest consensus state: current_term: 2 leader_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:29.040385 23618 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:29.040553 23620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:29.041002 23620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:29.041064 23620 catalog_manager.cc:1259] Loaded cluster ID: 0defa86d538b4ccc9c8e6e52a435bad3
I20260619 08:51:29.041117 23620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:29.041278 23620 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:29.041361 23620 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Loaded TSK: 0
I20260619 08:51:29.041517 23620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:51:29.093026 23088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:29.094687 23088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:29.095762 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.095643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:52084)
0619 08:51:29.095702 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.095736 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 08:51:29.095790 22912 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [term 2 FOLLOWER]: Refusing update from remote peer a88ac38d150b425c9c1ddd9fe524fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260619 08:51:29.096092 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.095990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.0.0.1:52088)
0619 08:51:29.096034 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.096067 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 08:51:29.096118 23618 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260619 08:51:29.096247 23072 consensus_peers.cc:487] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 -> Peer da6a78a0b5bb473aa819bb6ce926b5b2 (127.15.208.62:46545): Couldn't send request to peer da6a78a0b5bb473aa819bb6ce926b5b2. Status: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 08:51:29.096765 23609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a88ac38d150b425c9c1ddd9fe524fa12. Latest consensus state: current_term: 2 leader_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:29.096876 23609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:51:29.097275 23608 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:29.097358 23608 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:29.097505 23609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da6a78a0b5bb473aa819bb6ce926b5b2" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46545 } } peers { permanent_uuid: "0806610f714f4ed295d35650ab23e71d" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34855 } attrs { promote: false } } peers { permanent_uuid: "a88ac38d150b425c9c1ddd9fe524fa12" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 35615 } attrs { promote: false } } }
I20260619 08:51:29.097604 23609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0806610f714f4ed295d35650ab23e71d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:51:29.102293 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.098968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:46255 (local address 127.0.0.1:39284)
0619 08:51:29.099085 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.099097 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:29.099140 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:29.099526 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:29.099526 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:29.099528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:29.099560 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.099561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.100481 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.100482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.101332 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.101334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.102083 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.102084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.102144 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.102148 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:29.102177 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 08:51:29.102216 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":1}
I20260619 08:51:29.102596 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.098911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:36031 (local address 127.0.0.1:46418)
0619 08:51:29.098939 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.098950 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:29.098998 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:29.099335 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:29.099336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:29.099338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:29.099393 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.099395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.100364 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.100365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.101647 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.101650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.102355 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.102356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.102417 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.102420 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:29.102473 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:51:29.102522 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14,"mutex_wait_us":4}
I20260619 08:51:29.102620 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.099088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46418 (local address 127.15.208.1:36031)
0619 08:51:29.099204 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:29.099206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:29.099214 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:29.099244 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:29.099245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:29.099256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:29.099315 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:29.099474 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.100253 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.101730 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.102269 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.102339 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.102353 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:29.102495 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 08:51:29.102558 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260619 08:51:29.102764 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.098981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.15.208.3:45669)
0619 08:51:29.099291 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:29.099292 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:29.099700 (+ 408us) server_negotiation.cc:407] Connection header received
0619 08:51:29.099732 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:29.099732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:29.099740 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:29.099807 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:29.099939 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.100636 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.101894 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.102287 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.102374 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.102386 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:29.102626 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 08:51:29.102674 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":67,"threads_started":1}
I20260619 08:51:29.102864 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.098916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:45669 (local address 127.0.0.1:38040)
0619 08:51:29.099579 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.099589 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:29.099638 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:29.099826 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:29.099827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:29.099829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:29.099871 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.099872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.100762 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.100763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.101810 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.101812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.102414 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.102415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.102474 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.102478 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:29.102522 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 08:51:29.102552 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":138,"thread_start_us":71,"threads_started":1}
I20260619 08:51:29.102865 23201 tablet_service.cc:1467] Processing CreateTablet for tablet 84780a96d7744e06855851d3f76a34fc (DEFAULT_TABLE table=test-table [id=f437156334c54a7e80c11ba0f2d24baa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:29.103003 23201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84780a96d7744e06855851d3f76a34fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:29.103220 23474 tablet_service.cc:1467] Processing CreateTablet for tablet 84780a96d7744e06855851d3f76a34fc (DEFAULT_TABLE table=test-table [id=f437156334c54a7e80c11ba0f2d24baa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:29.103273 23628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.099076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39284 (local address 127.15.208.2:46255)
0619 08:51:29.099389 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:29.099391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:29.099396 (+ 5us) server_negotiation.cc:407] Connection header received
0619 08:51:29.099435 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:29.099435 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:29.099446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:29.099512 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:29.099641 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.100339 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.101426 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.101980 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.103012 (+ 1032us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.103025 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:29.103043 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 08:51:29.103193 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":54,"threads_started":1}
I20260619 08:51:29.103360 23474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84780a96d7744e06855851d3f76a34fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:29.104368 23630 tablet_bootstrap.cc:492] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45: Bootstrap starting.
I20260619 08:51:29.104632 23338 tablet_service.cc:1467] Processing CreateTablet for tablet 84780a96d7744e06855851d3f76a34fc (DEFAULT_TABLE table=test-table [id=f437156334c54a7e80c11ba0f2d24baa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:29.104799 23630 tablet_bootstrap.cc:654] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:29.104808 23338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84780a96d7744e06855851d3f76a34fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:29.104856 23631 tablet_bootstrap.cc:492] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3: Bootstrap starting.
I20260619 08:51:29.105214 23631 tablet_bootstrap.cc:654] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:29.105409 23630 tablet_bootstrap.cc:492] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45: No bootstrap required, opened a new log
I20260619 08:51:29.105455 23630 ts_tablet_manager.cc:1397] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:29.105566 23630 raft_consensus.cc:348] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } } peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } }
I20260619 08:51:29.105616 23630 raft_consensus.cc:374] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:29.105629 23630 raft_consensus.cc:729] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 851148a914324cc5ba28a76da4424c45, State: Initialized, Role: FOLLOWER
I20260619 08:51:29.105710 23630 consensus_queue.cc:260] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } } peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } }
I20260619 08:51:29.105832 23631 tablet_bootstrap.cc:492] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3: No bootstrap required, opened a new log
I20260619 08:51:29.105863 23630 ts_tablet_manager.cc:1428] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:29.105871 23631 ts_tablet_manager.cc:1397] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:29.106148 23631 raft_consensus.cc:348] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } } peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } }
I20260619 08:51:29.106220 23631 raft_consensus.cc:374] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:29.106238 23631 raft_consensus.cc:729] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8768e3237994432af741ad63b27d8d3, State: Initialized, Role: FOLLOWER
I20260619 08:51:29.106294 23631 consensus_queue.cc:260] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } } peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } }
I20260619 08:51:29.106441 23631 ts_tablet_manager.cc:1428] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:29.106524 23267 heartbeater.cc:503] Master 127.15.208.60:35615 was elected leader, sending a full tablet report...
I20260619 08:51:29.106812 23635 tablet_bootstrap.cc:492] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a: Bootstrap starting.
I20260619 08:51:29.107187 23635 tablet_bootstrap.cc:654] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:29.107527 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.106665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.3:48699)
0619 08:51:29.107426 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.107482 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":76,"threads_started":1}
I20260619 08:51:29.107870 23635 tablet_bootstrap.cc:492] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a: No bootstrap required, opened a new log
I20260619 08:51:29.107914 23635 ts_tablet_manager.cc:1397] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:29.108001 23635 raft_consensus.cc:348] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } } peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } }
I20260619 08:51:29.108052 23635 raft_consensus.cc:374] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:29.108067 23635 raft_consensus.cc:729] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc40bdaed75848d2b8a9d58ec180887a, State: Initialized, Role: FOLLOWER
I20260619 08:51:29.108122 23635 consensus_queue.cc:260] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } } peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } }
I20260619 08:51:29.108371 23635 ts_tablet_manager.cc:1428] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:29.108475 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.106466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.1:51457)
0619 08:51:29.107664 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.107701 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1178,"thread_start_us":82,"threads_started":1}
I20260619 08:51:29.108673 23404 heartbeater.cc:503] Master 127.15.208.60:35615 was elected leader, sending a full tablet report...
I20260619 08:51:29.108784 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.108409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.2:49563)
0619 08:51:29.108675 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.108713 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":76,"threads_started":1}
I20260619 08:51:29.109112 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.108985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.2:50773)
0619 08:51:29.109060 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.109088 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 08:51:29.119412 23636 raft_consensus.cc:482] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:29.119495 23636 raft_consensus.cc:504] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } } peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } }
I20260619 08:51:29.119889 23636 leader_election.cc:288] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 851148a914324cc5ba28a76da4424c45 (127.15.208.1:36031), bc40bdaed75848d2b8a9d58ec180887a (127.15.208.2:46255)
I20260619 08:51:29.123005 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.120147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:45343 (local address 127.15.208.1:36031)
0619 08:51:29.120228 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:29.120230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:29.120240 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:29.120266 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:29.120267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:29.120276 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:29.120358 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:29.120584 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.121228 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.122272 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.122669 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.122723 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.122731 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:29.122892 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 08:51:29.122933 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 08:51:29.123008 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.120050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:36031 (local address 127.15.208.3:45343)
0619 08:51:29.120111 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.120123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:29.120200 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:29.120364 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:29.120365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:29.120367 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:29.120466 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.120468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.121306 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.121307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.122184 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.122187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.122744 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.122745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.122819 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.122824 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:51:29.122875 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 08:51:29.122916 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":33}
I20260619 08:51:29.123080 23628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.120144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:45215 (local address 127.15.208.2:46255)
0619 08:51:29.120230 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:29.120231 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:29.120472 (+ 241us) server_negotiation.cc:407] Connection header received
0619 08:51:29.120756 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:29.120757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:29.120766 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:29.120822 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:29.120954 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.121481 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.122387 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.122750 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.122819 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.122827 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:29.122980 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 08:51:29.123014 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 08:51:29.123136 23642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.120050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:46255 (local address 127.15.208.3:45215)
0619 08:51:29.120399 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.120410 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:29.120616 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:29.120844 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:29.120844 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:29.120846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:29.120888 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.120889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.121558 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.121559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.122305 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:29.122307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.122827 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.122828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:29.122904 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.122908 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:29.122964 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 08:51:29.123000 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":142,"thread_start_us":124,"threads_started":1}
I20260619 08:51:29.123443 23358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84780a96d7744e06855851d3f76a34fc" candidate_uuid: "e8768e3237994432af741ad63b27d8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc40bdaed75848d2b8a9d58ec180887a" is_pre_election: true
I20260619 08:51:29.123541 23358 raft_consensus.cc:2454] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8768e3237994432af741ad63b27d8d3 in term 0.
I20260619 08:51:29.123531 23221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84780a96d7744e06855851d3f76a34fc" candidate_uuid: "e8768e3237994432af741ad63b27d8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "851148a914324cc5ba28a76da4424c45" is_pre_election: true
I20260619 08:51:29.123683 23221 raft_consensus.cc:2454] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8768e3237994432af741ad63b27d8d3 in term 0.
I20260619 08:51:29.123708 23429 leader_election.cc:302] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc40bdaed75848d2b8a9d58ec180887a, e8768e3237994432af741ad63b27d8d3; no voters:
I20260619 08:51:29.123826 23636 raft_consensus.cc:2781] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:29.123871 23636 raft_consensus.cc:482] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:29.123889 23636 raft_consensus.cc:3037] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:29.124555 23636 raft_consensus.cc:504] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } } peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } }
I20260619 08:51:29.124694 23636 leader_election.cc:288] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [CANDIDATE]: Term 1 election: Requested vote from peers 851148a914324cc5ba28a76da4424c45 (127.15.208.1:36031), bc40bdaed75848d2b8a9d58ec180887a (127.15.208.2:46255)
I20260619 08:51:29.124861 23221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84780a96d7744e06855851d3f76a34fc" candidate_uuid: "e8768e3237994432af741ad63b27d8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "851148a914324cc5ba28a76da4424c45"
I20260619 08:51:29.124910 23358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84780a96d7744e06855851d3f76a34fc" candidate_uuid: "e8768e3237994432af741ad63b27d8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc40bdaed75848d2b8a9d58ec180887a"
I20260619 08:51:29.124965 23221 raft_consensus.cc:3037] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:29.124990 23358 raft_consensus.cc:3037] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:29.125720 23358 raft_consensus.cc:2454] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8768e3237994432af741ad63b27d8d3 in term 1.
I20260619 08:51:29.125859 23221 raft_consensus.cc:2454] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8768e3237994432af741ad63b27d8d3 in term 1.
I20260619 08:51:29.125895 23429 leader_election.cc:302] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc40bdaed75848d2b8a9d58ec180887a, e8768e3237994432af741ad63b27d8d3; no voters:
I20260619 08:51:29.125972 23636 raft_consensus.cc:2781] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:29.126118 23636 raft_consensus.cc:686] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 1 LEADER]: Becoming Leader. State: Replica: e8768e3237994432af741ad63b27d8d3, State: Running, Role: LEADER
I20260619 08:51:29.126207 23540 heartbeater.cc:503] Master 127.15.208.60:35615 was elected leader, sending a full tablet report...
I20260619 08:51:29.126235 23636 consensus_queue.cc:237] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } } peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } }
I20260619 08:51:29.126407 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.126313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.3:57427)
0619 08:51:29.126343 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.126375 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260619 08:51:29.126917 23088 catalog_manager.cc:5495] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8768e3237994432af741ad63b27d8d3 (127.15.208.3). New cstate: current_term: 1 leader_uuid: "e8768e3237994432af741ad63b27d8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8768e3237994432af741ad63b27d8d3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 45669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:29.138689 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.136386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38048 (local address 127.15.208.3:45669)
0619 08:51:29.136449 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:29.136451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:29.136459 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:29.136528 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:29.136528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:29.136544 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:29.136602 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:29.136711 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.137282 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.138002 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:29.138196 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:29.138247 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:29.138379 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:29.138476 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:29.138528 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:29.138572 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:51:29.138588 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:51:29.139863 23358 raft_consensus.cc:1261] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a [term 1 FOLLOWER]: Refusing update from remote peer e8768e3237994432af741ad63b27d8d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:29.139935 23221 raft_consensus.cc:1261] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45 [term 1 FOLLOWER]: Refusing update from remote peer e8768e3237994432af741ad63b27d8d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:29.140146 23643 consensus_queue.cc:1035] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc40bdaed75848d2b8a9d58ec180887a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:29.140264 23636 consensus_queue.cc:1035] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "851148a914324cc5ba28a76da4424c45" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 36031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:29.141397 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.141143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.1:33457)
0619 08:51:29.141322 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.141361 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260619 08:51:29.141449 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.141326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.2:39531)
0619 08:51:29.141385 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.141416 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:51:29.142431 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.142327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.3:48879)
0619 08:51:29.142365 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.142399 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 08:51:29.142704 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.142521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.2:54083)
0619 08:51:29.142633 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.142673 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260619 08:51:29.143026 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.142763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.3:56439)
0619 08:51:29.142953 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.142997 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260619 08:51:29.143217 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.143115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46545 (local address 127.15.208.1:46977)
0619 08:51:29.143152 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.143190 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:46545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 08:51:29.144886 23087 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:29.144965 23087 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:29.146155 23087 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a88ac38d150b425c9c1ddd9fe524fa12: Sending DeleteTablet for 3 replicas of tablet 84780a96d7744e06855851d3f76a34fc
I20260619 08:51:29.146443 23201 tablet_service.cc:1514] Processing DeleteTablet for tablet 84780a96d7744e06855851d3f76a34fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:29 UTC) from {username='oryx', principal='oryx/127.15.208.60@KRBTEST.COM'} at 127.0.0.1:46418
I20260619 08:51:29.146476 23338 tablet_service.cc:1514] Processing DeleteTablet for tablet 84780a96d7744e06855851d3f76a34fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:29 UTC) from {username='oryx', principal='oryx/127.15.208.60@KRBTEST.COM'} at 127.0.0.1:39284
I20260619 08:51:29.146638 23656 tablet_replica.cc:330] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45: stopping tablet replica
I20260619 08:51:29.146660 23474 tablet_service.cc:1514] Processing DeleteTablet for tablet 84780a96d7744e06855851d3f76a34fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:29 UTC) from {username='oryx', principal='oryx/127.15.208.60@KRBTEST.COM'} at 127.0.0.1:38040
I20260619 08:51:29.146696 23656 raft_consensus.cc:2229] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:29.146749 23656 raft_consensus.cc:2258] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:29.146867 23658 tablet_replica.cc:330] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3: stopping tablet replica
I20260619 08:51:29.146961 23658 raft_consensus.cc:2229] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:29.147073 23658 raft_consensus.cc:2258] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:29.147066 23656 ts_tablet_manager.cc:1905] T 84780a96d7744e06855851d3f76a34fc P 851148a914324cc5ba28a76da4424c45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:29.147181 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 23049
I20260619 08:51:29.147449 23658 ts_tablet_manager.cc:1905] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:29.149179 23658 ts_tablet_manager.cc:1918] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:29.149307 23658 log.cc:1196] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/84780a96d7744e06855851d3f76a34fc
I20260619 08:51:29.149612 23658 ts_tablet_manager.cc:1939] T 84780a96d7744e06855851d3f76a34fc P e8768e3237994432af741ad63b27d8d3: Deleting consensus metadata
I20260619 08:51:29.150024 23073 catalog_manager.cc:4841] TS e8768e3237994432af741ad63b27d8d3 (127.15.208.3:45669): tablet 84780a96d7744e06855851d3f76a34fc (table test-table [id=f437156334c54a7e80c11ba0f2d24baa]) successfully deleted
I20260619 08:51:29.152865 23657 tablet_replica.cc:330] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a: stopping tablet replica
I20260619 08:51:29.152936 23657 raft_consensus.cc:2229] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:29.152992 23657 raft_consensus.cc:2258] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:29.153337 23657 ts_tablet_manager.cc:1905] T 84780a96d7744e06855851d3f76a34fc P bc40bdaed75848d2b8a9d58ec180887a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 08:51:29.154136 23627 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:29.153895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:36031 (local address 127.0.0.1:46428)
0619 08:51:29.153940 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.153951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:29.153986 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:29.154089 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:36031: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 08:51:29.154351 23075 catalog_manager.cc:4584] TS 851148a914324cc5ba28a76da4424c45 (127.15.208.1:36031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 84780a96d7744e06855851d3f76a34fc: Network error: Client connection negotiation failed: client connection to 127.15.208.1:36031: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:51:29.154376 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 23275
I20260619 08:51:29.159617 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:29.159491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:46255 (local address 127.0.0.1:39286)
0619 08:51:29.159552 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:29.159581 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.2:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 08:51:29.160118 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 23411
I20260619 08:51:29.164647 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 22768
I20260619 08:51:29.169200 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 22956
2026-06-19T08:51:29Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4634 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:29 dist-test-slave-p0h9 krb5kdc[23661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:29 dist-test-slave-p0h9 krb5kdc[23661](info): set up 2 sockets
Jun 19 08:51:29 dist-test-slave-p0h9 krb5kdc[23661](info): commencing operation
krb5kdc: starting...
W20260619 08:51:30.209825 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.62.keytab.
Entry for principal oryx/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.62.keytab.
I20260619 08:51:30.327872 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:43837
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.62.keytab
--principal=oryx/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:43837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:30.411942 23674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:30.412159 23674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:30.412182 23674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:30.413358 23674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:30.413398 23674 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 08:51:30.413412 23674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:30.413425 23674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:30.413439 23674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:30.413450 23674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:30.414861 23674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:43837
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.62.keytab
--principal=oryx/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:43837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:30.415059 23674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:30.415223 23674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:30.419976 23679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:30.420003 23680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:30.419989 23682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:30.420257 23674 server_base.cc:1034] running on GCE node
I20260619 08:51:30.420514 23674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:30.420862 23674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:30.422032 23674 hybrid_clock.cc:648] HybridClock initialized: now 1781859090422012 us; error 34 us; skew 500 ppm
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:30.424539 23674 init.cc:377] Logged in from keytab as oryx/127.15.208.62@KRBTEST.COM (short username oryx)
I20260619 08:51:30.425074 23674 webserver.cc:460] Webserver started at http://127.15.208.62:42663/ using document root <none> and password file <none>
I20260619 08:51:30.425295 23674 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:30.425360 23674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:30.425515 23674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:30.426625 23674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "3ee8b79298cc4777b162bbd23fb5d01f"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "56b8c9af080930427e66d6a8e00a327e"
server_key_iv: "282d097dd15cb68db2990a129e9412b2"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.427028 23674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "3ee8b79298cc4777b162bbd23fb5d01f"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "56b8c9af080930427e66d6a8e00a327e"
server_key_iv: "282d097dd15cb68db2990a129e9412b2"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.428390 23674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:30.429200 23689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:30.429479 23674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:51:30.429623 23674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "3ee8b79298cc4777b162bbd23fb5d01f"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "56b8c9af080930427e66d6a8e00a327e"
server_key_iv: "282d097dd15cb68db2990a129e9412b2"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.429703 23674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:30.442054 23674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:30.444484 23674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:30.444664 23674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:30.449349 23674 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:43837
I20260619 08:51:30.449431 23751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:43837 every 8 connection(s)
I20260619 08:51:30.449779 23674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:30.450474 23752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:30.452553 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 23674
I20260619 08:51:30.452644 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:30.452847 23752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f: Bootstrap starting.
I20260619 08:51:30.452873 16192 external_mini_cluster.cc:1428] Setting key 7c92e38522231a68544cfc82ca201854
I20260619 08:51:30.453542 23752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:30.453835 23752 log.cc:824] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:30.454571 23752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f: No bootstrap required, opened a new log
I20260619 08:51:30.456347 23752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee8b79298cc4777b162bbd23fb5d01f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43837 } }
I20260619 08:51:30.456509 23752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:30.456560 23752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ee8b79298cc4777b162bbd23fb5d01f, State: Initialized, Role: FOLLOWER
I20260619 08:51:30.456707 23752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee8b79298cc4777b162bbd23fb5d01f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43837 } }
I20260619 08:51:30.456818 23752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:30.456847 23752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:30.456890 23752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:30.457568 23752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee8b79298cc4777b162bbd23fb5d01f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43837 } }
I20260619 08:51:30.457686 23752 leader_election.cc:302] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ee8b79298cc4777b162bbd23fb5d01f; no voters:
I20260619 08:51:30.457860 23752 leader_election.cc:288] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:30.457901 23757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:30.458081 23757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [term 1 LEADER]: Becoming Leader. State: Replica: 3ee8b79298cc4777b162bbd23fb5d01f, State: Running, Role: LEADER
I20260619 08:51:30.458129 23752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:30.458187 23757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee8b79298cc4777b162bbd23fb5d01f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43837 } }
I20260619 08:51:30.458484 23757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ee8b79298cc4777b162bbd23fb5d01f. Latest consensus state: current_term: 1 leader_uuid: "3ee8b79298cc4777b162bbd23fb5d01f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee8b79298cc4777b162bbd23fb5d01f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43837 } } }
I20260619 08:51:30.458564 23757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:30.458500 23759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ee8b79298cc4777b162bbd23fb5d01f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee8b79298cc4777b162bbd23fb5d01f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43837 } } }
I20260619 08:51:30.458603 23759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:30.459487 23761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:30.459966 23761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:30.461565 23761 catalog_manager.cc:1347] Generated new cluster ID: 3abebfab968b4983ae93a6e8f76b839d
I20260619 08:51:30.461619 23761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:30.464071 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.453363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44594 (local address 127.15.208.62:43837)
0619 08:51:30.453565 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.453575 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.453607 (+ 32us) server_negotiation.cc:407] Connection header received
0619 08:51:30.454119 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.454127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.454175 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.454319 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:30.455006 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.455850 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.456645 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.457277 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.457410 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.460424 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:30.460428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:30.460428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:30.460439 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:30.462426 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.462826 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.462829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.462830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.462936 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.463147 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.463148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.463148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.463461 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:30.463609 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.463717 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.463852 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260619 08:51:30.482654 23761 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:30.483222 23761 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:30.487301 23761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f: Generated new TSK 0
I20260619 08:51:30.487528 23761 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab.
Entry for principal oryx/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab.
I20260619 08:51:30.515945 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:43837
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab
--principal=oryx/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:30.598492 23778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:30.598721 23778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:30.598765 23778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:30.599936 23778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:30.600003 23778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:30.600040 23778 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:51:30.600106 23778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:30.601740 23778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab
--principal=oryx/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.208.62:43837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:30.602025 23778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:30.602262 23778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:30.607272 23786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:30.607273 23783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:30.607295 23784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:30.607789 23778 server_base.cc:1034] running on GCE node
I20260619 08:51:30.608012 23778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:30.608305 23778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:30.609484 23778 hybrid_clock.cc:648] HybridClock initialized: now 1781859090609465 us; error 51 us; skew 500 ppm
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:30.612140 23778 init.cc:377] Logged in from keytab as oryx/127.15.208.1@KRBTEST.COM (short username oryx)
I20260619 08:51:30.612545 23778 webserver.cc:460] Webserver started at http://127.15.208.1:34157/ using document root <none> and password file <none>
I20260619 08:51:30.612850 23778 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:30.612944 23778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:30.613137 23778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:30.614230 23778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "588543e446414f85ae08087399584a6f"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "b664272d9c0db8b759cab3eff6db0505"
server_key_iv: "21c011ca399aa6084c7bcd1d8dac72a4"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.614598 23778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "588543e446414f85ae08087399584a6f"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "b664272d9c0db8b759cab3eff6db0505"
server_key_iv: "21c011ca399aa6084c7bcd1d8dac72a4"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.615955 23778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:30.616844 23793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:30.617065 23778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:30.617174 23778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "588543e446414f85ae08087399584a6f"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "b664272d9c0db8b759cab3eff6db0505"
server_key_iv: "21c011ca399aa6084c7bcd1d8dac72a4"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.617268 23778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:30.632583 23778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:30.634879 23778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:30.635056 23778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:30.635558 23778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:30.635624 23778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:30.635665 23778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:30.635695 23778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:30.643110 23778 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:43335
I20260619 08:51:30.643193 23908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:43335 every 8 connection(s)
I20260619 08:51:30.643513 23778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:30.646427 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.636096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:37775 (local address 127.15.208.62:43837)
0619 08:51:30.636167 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.636170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.636415 (+ 245us) server_negotiation.cc:407] Connection header received
0619 08:51:30.637147 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.637148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.637157 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.637200 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:30.638120 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.638972 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.640692 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.641058 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.641134 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.643639 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:30.643641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:30.643641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:30.643648 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:30.645073 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.645458 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.645460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.645460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.645522 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.645769 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.645770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.645770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.645896 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:30.645966 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.646240 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.646324 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:51:30.646610 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.636067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43837 (local address 127.15.208.1:37775)
0619 08:51:30.636273 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:30.636321 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:30.637038 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:30.637284 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:30.637289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:30.637689 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:30.637990 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.637997 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.639067 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.639068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.640578 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.640581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.641145 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.641146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.641308 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.641894 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:30.641904 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:30.643521 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:30.645164 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.645170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.645172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.645382 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.645591 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.645592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.645592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.645697 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.645970 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:30.645976 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:30.646108 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 08:51:30.646206 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 08:51:30.646355 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260619 08:51:30.647140 23909 heartbeater.cc:347] Connected to a master server at 127.15.208.62:43837
I20260619 08:51:30.647224 23909 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:30.647403 23909 heartbeater.cc:511] Master 127.15.208.62:43837 requested a full tablet report, sending...
I20260619 08:51:30.648002 23705 ts_manager.cc:194] Registered new tserver with Master: 588543e446414f85ae08087399584a6f (127.15.208.1:43335)
I20260619 08:51:30.648947 23705 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:37775
I20260619 08:51:30.651129 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 23778
I20260619 08:51:30.651248 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:30.651540 16192 external_mini_cluster.cc:1428] Setting key 9c4e0d07b627929d73e099c5dcf12f2f
I20260619 08:51:30.653847 23915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.648149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43837 (local address 127.15.208.1:50613)
0619 08:51:30.648286 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:30.648296 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:30.648356 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:30.648470 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:30.648470 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:30.648662 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:30.648716 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.648718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.649469 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.649470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.650086 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.650088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.650396 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.650396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.650441 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.650781 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:30.650785 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:30.651049 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:30.652819 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.652820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.652821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.652990 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.653197 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.653198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.653198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.653271 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.653549 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:30.653550 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:30.653607 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 08:51:30.653689 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:51:30.653780 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260619 08:51:30.653937 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.648237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:50613 (local address 127.15.208.62:43837)
0619 08:51:30.648275 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.648277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.648348 (+ 71us) server_negotiation.cc:407] Connection header received
0619 08:51:30.648410 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.648410 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.648418 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.648457 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:30.648798 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.649379 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.650160 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.650332 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.650385 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.651139 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:30.651140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:30.651140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:30.651147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:30.652704 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.653074 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.653075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.653075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.653133 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.653339 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.653340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.653340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.653466 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:30.653535 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.653729 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.653811 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for oryx/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab.
Entry for principal oryx/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab.
I20260619 08:51:30.676648 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:43837
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab
--principal=oryx/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:30.756842 23918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:30.757068 23918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:30.757105 23918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:30.758347 23918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:30.758406 23918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:30.758445 23918 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:51:30.758481 23918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:30.760095 23918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab
--principal=oryx/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.208.62:43837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:30.760355 23918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:30.760587 23918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:30.765607 23926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:30.765586 23924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:30.765628 23923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:30.765848 23918 server_base.cc:1034] running on GCE node
I20260619 08:51:30.766098 23918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:30.766350 23918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:30.767489 23918 hybrid_clock.cc:648] HybridClock initialized: now 1781859090767470 us; error 41 us; skew 500 ppm
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:30.769968 23918 init.cc:377] Logged in from keytab as oryx/127.15.208.2@KRBTEST.COM (short username oryx)
I20260619 08:51:30.770422 23918 webserver.cc:460] Webserver started at http://127.15.208.2:40395/ using document root <none> and password file <none>
I20260619 08:51:30.770602 23918 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:30.770643 23918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:30.770732 23918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:30.771802 23918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "5e3deadcc2574864af2d9c40468635e3"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "5f6cabae6369630991e579ae1dd27e23"
server_key_iv: "73a595ce592001b602a801526f3fb827"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.772112 23918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "5e3deadcc2574864af2d9c40468635e3"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "5f6cabae6369630991e579ae1dd27e23"
server_key_iv: "73a595ce592001b602a801526f3fb827"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.773422 23918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:30.774112 23933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:30.774282 23918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:30.774380 23918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "5e3deadcc2574864af2d9c40468635e3"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "5f6cabae6369630991e579ae1dd27e23"
server_key_iv: "73a595ce592001b602a801526f3fb827"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.774452 23918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:30.782267 23918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:30.784097 23918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:30.784250 23918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:30.784801 23918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:30.784855 23918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:30.784897 23918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:30.784927 23918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:30.792126 23918 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:35509
I20260619 08:51:30.792549 23918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:51:30.792826 24048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:35509 every 8 connection(s)
I20260619 08:51:30.794158 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.785238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:34027 (local address 127.15.208.62:43837)
0619 08:51:30.785299 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.785301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.785468 (+ 167us) server_negotiation.cc:407] Connection header received
0619 08:51:30.785980 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.785981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.785993 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.786051 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:30.786649 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.787153 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.788197 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.788408 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.788497 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.791124 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:30.791126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:30.791126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:30.791136 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:30.792967 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.793391 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.793392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.793393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.793450 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.793625 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.793626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.793626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.793754 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:30.793830 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.794023 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.794095 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:51:30.794332 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.785208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43837 (local address 127.15.208.2:34027)
0619 08:51:30.785356 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:30.785388 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:30.785874 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:30.786130 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:30.786133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:30.786373 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:30.786549 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.786554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.787262 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.787263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.788100 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.788102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.788504 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.788505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.788646 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.789208 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:30.789216 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:30.790986 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:30.793065 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.793070 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.793072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.793323 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.793498 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.793498 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.793498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.793587 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.793820 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:30.793823 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:30.793930 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 08:51:30.794007 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:51:30.794127 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260619 08:51:30.794998 24049 heartbeater.cc:347] Connected to a master server at 127.15.208.62:43837
I20260619 08:51:30.795081 24049 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:30.795279 24049 heartbeater.cc:511] Master 127.15.208.62:43837 requested a full tablet report, sending...
I20260619 08:51:30.795701 23706 ts_manager.cc:194] Registered new tserver with Master: 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509)
I20260619 08:51:30.796185 23706 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:34027
I20260619 08:51:30.801424 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 23918
I20260619 08:51:30.801473 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.796226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:36469 (local address 127.15.208.62:43837)
0619 08:51:30.796307 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.796309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.796374 (+ 65us) server_negotiation.cc:407] Connection header received
0619 08:51:30.796441 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.796441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.796451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.796515 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:30.796884 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.797352 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.798075 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.798281 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.798363 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.799190 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:30.799191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:30.799192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:30.799198 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:30.800286 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.800639 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.800640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.800641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.800694 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.800926 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.800927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.800928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.801080 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:30.801155 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.801319 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.801386 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 08:51:30.801488 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.796150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43837 (local address 127.15.208.2:36469)
0619 08:51:30.796318 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:30.796328 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:30.796385 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:30.796512 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:30.796513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:30.796702 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:30.796804 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.796806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.797444 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.797445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.797962 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.797964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.798365 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.798365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.798436 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.798830 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:30.798834 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:30.799104 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:30.800384 (+ 1280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.800385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.800385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.800556 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.800801 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.800802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.800802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.800858 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.801157 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:30.801158 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:30.801203 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:51:30.801281 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:51:30.801384 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260619 08:51:30.801584 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:30.801827 16192 external_mini_cluster.cc:1428] Setting key 7546818449434923bbcf538437f85409
WARNING: no policy specified for oryx/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab.
Entry for principal oryx/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab.
I20260619 08:51:30.825456 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:43837
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab
--principal=oryx/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:30.906509 24058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:30.906719 24058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:30.906744 24058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:30.907959 24058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:30.908008 24058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:30.908030 24058 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:51:30.908047 24058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:30.909505 24058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab
--principal=oryx/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.208.62:43837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:30.909683 24058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:30.909850 24058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:30.914633 24064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:30.914633 24063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:30.914646 24066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:30.914862 24058 server_base.cc:1034] running on GCE node
I20260619 08:51:30.915177 24058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:30.915376 24058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:30.916561 24058 hybrid_clock.cc:648] HybridClock initialized: now 1781859090916539 us; error 33 us; skew 500 ppm
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:30.919050 24058 init.cc:377] Logged in from keytab as oryx/127.15.208.3@KRBTEST.COM (short username oryx)
I20260619 08:51:30.919538 24058 webserver.cc:460] Webserver started at http://127.15.208.3:36401/ using document root <none> and password file <none>
I20260619 08:51:30.919719 24058 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:30.919759 24058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:30.919847 24058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:30.921083 24058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "1676542e561c4d6ea4d0b91225c821ba"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "4e2a8498cd0753ce87fc73f1ee65e3d6"
server_key_iv: "ae79675f4214896757b740aa72456b23"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.921393 24058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "1676542e561c4d6ea4d0b91225c821ba"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "4e2a8498cd0753ce87fc73f1ee65e3d6"
server_key_iv: "ae79675f4214896757b740aa72456b23"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.922689 24058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:30.923377 24073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:30.923547 24058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:30.923636 24058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "1676542e561c4d6ea4d0b91225c821ba"
format_stamp: "Formatted at 2026-06-19 08:51:30 on dist-test-slave-p0h9"
server_key: "4e2a8498cd0753ce87fc73f1ee65e3d6"
server_key_iv: "ae79675f4214896757b740aa72456b23"
server_key_version: "encryptionkey@0"
I20260619 08:51:30.923691 24058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:30.934495 24058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:30.936290 24058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:30.936398 24058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:30.936921 24058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:30.936985 24058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:30.937021 24058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:30.937047 24058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:30.944434 24058 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:39075
I20260619 08:51:30.944481 24188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:39075 every 8 connection(s)
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:30.944875 24058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:30.948326 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.937360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:38015 (local address 127.15.208.62:43837)
0619 08:51:30.937402 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.937404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.937669 (+ 265us) server_negotiation.cc:407] Connection header received
0619 08:51:30.938205 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.938206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.938217 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.938290 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:30.939240 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.940117 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.942147 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.942471 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.942528 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.945355 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:30.945356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:30.945357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:30.945363 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:30.946910 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.947548 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.947548 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.947549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.947601 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.947820 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.947820 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.947820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.947920 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:30.947969 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.948188 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.948258 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:51:30.948541 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.937325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43837 (local address 127.15.208.3:38015)
0619 08:51:30.937516 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:30.937570 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:30.938098 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:30.938422 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:30.938428 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:30.938812 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:30.939129 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.939137 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.940229 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.940230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.942032 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.942036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.942553 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.942554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.942722 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.943345 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:30.943356 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:30.945269 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:30.947143 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.947149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.947151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.947474 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.947670 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.947671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.947671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.947766 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.947978 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:30.947981 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:30.948085 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 08:51:30.948172 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 08:51:30.948298 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260619 08:51:30.949059 24189 heartbeater.cc:347] Connected to a master server at 127.15.208.62:43837
I20260619 08:51:30.949141 24189 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:30.949327 24189 heartbeater.cc:511] Master 127.15.208.62:43837 requested a full tablet report, sending...
I20260619 08:51:30.949715 23706 ts_manager.cc:194] Registered new tserver with Master: 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075)
I20260619 08:51:30.949967 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 24058
I20260619 08:51:30.950129 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:30.950255 23706 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:38015
I20260619 08:51:30.950376 16192 external_mini_cluster.cc:1428] Setting key 6400aeb2e72d79e4add659dbc44fc9fc
I20260619 08:51:30.951812 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:51:30.956372 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.950395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:39069 (local address 127.15.208.62:43837)
0619 08:51:30.950440 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.950442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.950453 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:51:30.950521 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.950522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.950532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.950590 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:30.950929 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.951612 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.952467 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.952799 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.952869 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.953779 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:30.953780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:30.953781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:30.953789 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:30.955247 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.955576 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.955577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.955578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.955631 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.955856 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.955857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.955857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.955966 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:30.956042 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.956208 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.956281 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:51:30.956382 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.950172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43837 (local address 127.15.208.3:39069)
0619 08:51:30.950367 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:30.950380 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:30.950464 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:30.950596 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:30.950597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:30.950796 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:30.950855 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.950857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.951709 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.951710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.952387 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.952389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.952872 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.952873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.952924 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.953356 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:30.953360 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:30.953655 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:30.955341 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.955342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.955343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.955505 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.955710 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:30.955710 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:30.955711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:30.955766 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:30.956033 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:30.956034 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:30.956079 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:51:30.956182 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 08:51:30.956284 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 08:51:30.973618 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:30.968409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.15.208.62:43837)
0619 08:51:30.968468 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.968470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.968478 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:30.968541 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.968542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.968550 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.968600 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:30.969160 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.969642 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.970352 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.970552 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.970607 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.973490 (+ 2883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44606: BlockingRecv error: recv got EOF from 127.0.0.1:44606 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 08:51:30 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:30.980628 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.974286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.15.208.62:43837)
0619 08:51:30.974330 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.974332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.974376 (+ 44us) server_negotiation.cc:407] Connection header received
0619 08:51:30.974429 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.974429 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.974436 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.974481 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:30.974913 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.975351 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.975979 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.976153 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.976202 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.978342 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:30.978343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:30.978343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:30.978349 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:30.979748 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.980037 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.980038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.980038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.980085 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:30.980227 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:30.980227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:30.980227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:30.980314 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:30.980381 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.980459 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.980526 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:51:30.982072 23706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:30.983621 23706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:30.989980 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.985887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.15.208.2:35509)
0619 08:51:30.986185 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.986191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.986202 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:51:30.986245 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.986247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.986265 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.986348 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:30.986603 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.987740 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.988849 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.989591 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.989670 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.989748 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.989875 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.989905 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":99,"threads_started":1}
I20260619 08:51:30.989982 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.985782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:35509 (local address 127.0.0.1:39092)
0619 08:51:30.986001 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:30.986052 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:30.986123 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:30.986432 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:30.986435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:30.986438 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:30.986505 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.986507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.987865 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.987866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.988780 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.988782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.989678 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.989679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.989765 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.989779 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:51:30.989857 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:51:30.989907 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":25,"thread_start_us":67,"threads_started":1}
I20260619 08:51:30.990327 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.985782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:39075 (local address 127.0.0.1:33828)
0619 08:51:30.986356 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:30.986367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:30.986417 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:30.986607 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:30.986608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:30.986609 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:30.986667 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.986668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.987617 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.987618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.988828 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.988831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.990099 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.990100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.990159 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.990163 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:30.990213 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:51:30.990249 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260619 08:51:30.990517 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.986176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53114 (local address 127.15.208.1:43335)
0619 08:51:30.986342 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.986346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.986807 (+ 461us) server_negotiation.cc:407] Connection header received
0619 08:51:30.986852 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.986854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.986870 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.986945 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:30.987073 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.988088 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.989380 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.990028 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.990093 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.990160 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.990369 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.990412 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260619 08:51:30.990649 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.985977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43335 (local address 127.0.0.1:53114)
0619 08:51:30.986698 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:30.986708 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:30.986753 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:30.986960 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:30.986961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:30.986964 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:30.987007 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.987008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.988166 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.988167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.989295 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:30.989297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.990192 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.990193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:30.990246 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.990248 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:30.990293 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 08:51:30.990321 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":43,"threads_started":1}
I20260619 08:51:30.990860 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:30.985975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33828 (local address 127.15.208.3:39075)
0619 08:51:30.986130 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:30.986136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:30.986471 (+ 335us) server_negotiation.cc:407] Connection header received
0619 08:51:30.986514 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:30.986517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:30.986532 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:30.986609 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:30.986744 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.987538 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.988898 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:30.989600 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:30.989670 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:30.989750 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:30.990439 (+ 689us) server_negotiation.cc:299] Negotiation successful
0619 08:51:30.990770 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260619 08:51:30.990789 23983 tablet_service.cc:1467] Processing CreateTablet for tablet 6a6345f10f8940b88f7ff191eed3503a (DEFAULT_TABLE table=test-table [id=dd85bce05ccd46c7a98e2072f34e0811]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:30.991195 23983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a6345f10f8940b88f7ff191eed3503a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:30.991441 24123 tablet_service.cc:1467] Processing CreateTablet for tablet 6a6345f10f8940b88f7ff191eed3503a (DEFAULT_TABLE table=test-table [id=dd85bce05ccd46c7a98e2072f34e0811]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:30.991760 24123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a6345f10f8940b88f7ff191eed3503a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:30.993464 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 6a6345f10f8940b88f7ff191eed3503a (DEFAULT_TABLE table=test-table [id=dd85bce05ccd46c7a98e2072f34e0811]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:30.993837 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a6345f10f8940b88f7ff191eed3503a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:30.994733 24215 tablet_bootstrap.cc:492] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: Bootstrap starting.
I20260619 08:51:30.995431 24215 tablet_bootstrap.cc:654] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:30.995744 24215 log.cc:824] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:30.996511 24215 tablet_bootstrap.cc:492] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: No bootstrap required, opened a new log
I20260619 08:51:30.996568 24215 ts_tablet_manager.cc:1397] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:30.996899 24214 tablet_bootstrap.cc:492] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: Bootstrap starting.
I20260619 08:51:30.997534 24214 tablet_bootstrap.cc:654] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:30.997560 24217 tablet_bootstrap.cc:492] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: Bootstrap starting.
I20260619 08:51:30.997902 24214 log.cc:824] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:30.998257 24217 tablet_bootstrap.cc:654] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:30.998576 24217 log.cc:824] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:30.998598 24215 raft_consensus.cc:348] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } }
I20260619 08:51:30.998767 24215 raft_consensus.cc:374] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:30.998816 24215 raft_consensus.cc:729] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1676542e561c4d6ea4d0b91225c821ba, State: Initialized, Role: FOLLOWER
I20260619 08:51:30.998839 24214 tablet_bootstrap.cc:492] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: No bootstrap required, opened a new log
I20260619 08:51:30.998901 24214 ts_tablet_manager.cc:1397] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:30.998929 24215 consensus_queue.cc:260] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } }
I20260619 08:51:30.999187 24215 ts_tablet_manager.cc:1428] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:51:30.999311 24189 heartbeater.cc:503] Master 127.15.208.62:43837 was elected leader, sending a full tablet report...
I20260619 08:51:30.999403 24217 tablet_bootstrap.cc:492] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: No bootstrap required, opened a new log
I20260619 08:51:30.999464 24217 ts_tablet_manager.cc:1397] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:31.000634 24214 raft_consensus.cc:348] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } }
I20260619 08:51:31.000815 24214 raft_consensus.cc:374] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.000862 24214 raft_consensus.cc:729] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e3deadcc2574864af2d9c40468635e3, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.000981 24214 consensus_queue.cc:260] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } }
I20260619 08:51:31.001058 24217 raft_consensus.cc:348] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } }
I20260619 08:51:31.001231 24217 raft_consensus.cc:374] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.001250 24214 ts_tablet_manager.cc:1428] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:51:31.001278 24217 raft_consensus.cc:729] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 588543e446414f85ae08087399584a6f, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.001374 24049 heartbeater.cc:503] Master 127.15.208.62:43837 was elected leader, sending a full tablet report...
I20260619 08:51:31.001390 24217 consensus_queue.cc:260] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } }
I20260619 08:51:31.001695 24217 ts_tablet_manager.cc:1428] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:51:31.001746 23909 heartbeater.cc:503] Master 127.15.208.62:43837 was elected leader, sending a full tablet report...
W20260619 08:51:31.043321 24050 tablet.cc:2369] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:51:31.058425 24220 raft_consensus.cc:482] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:31.058565 24220 raft_consensus.cc:504] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } }
I20260619 08:51:31.058909 24220 leader_election.cc:288] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509), 588543e446414f85ae08087399584a6f (127.15.208.1:43335)
I20260619 08:51:31.062062 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.059232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:60013 (local address 127.15.208.2:35509)
0619 08:51:31.059285 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.059287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.059364 (+ 77us) server_negotiation.cc:407] Connection header received
0619 08:51:31.059452 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.059453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.059465 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.059515 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:31.059734 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.060335 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.061292 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.061690 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.061745 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.061752 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.061933 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.061973 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:51:31.062073 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.059173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:35509 (local address 127.15.208.3:60013)
0619 08:51:31.059295 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:31.059323 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:31.059377 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:31.059570 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:31.059571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:31.059573 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:31.059650 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.059651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.060413 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.060414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.061206 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.061208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.061771 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.061772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.061861 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.061864 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:31.061924 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 08:51:31.061954 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260619 08:51:31.062158 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.059514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:53123 (local address 127.15.208.1:43335)
0619 08:51:31.059582 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.059584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.059594 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:31.059643 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.059644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.059651 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.059708 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:31.059878 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.060395 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.061385 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.061788 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.061862 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.061870 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.062042 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.062084 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:51:31.062394 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.059286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43335 (local address 127.15.208.3:53123)
0619 08:51:31.059525 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:31.059534 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:31.059577 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:31.059732 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:31.059733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:31.059735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:31.059797 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.059799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.060478 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.060479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.061297 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.061299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.061873 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.061874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.061959 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.061963 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:31.062019 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 08:51:31.062055 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":120,"threads_started":1}
I20260619 08:51:31.062454 24003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6345f10f8940b88f7ff191eed3503a" candidate_uuid: "1676542e561c4d6ea4d0b91225c821ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e3deadcc2574864af2d9c40468635e3" is_pre_election: true
I20260619 08:51:31.062644 24003 raft_consensus.cc:2454] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1676542e561c4d6ea4d0b91225c821ba in term 0.
I20260619 08:51:31.062693 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6345f10f8940b88f7ff191eed3503a" candidate_uuid: "1676542e561c4d6ea4d0b91225c821ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "588543e446414f85ae08087399584a6f" is_pre_election: true
I20260619 08:51:31.062860 23863 raft_consensus.cc:2454] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1676542e561c4d6ea4d0b91225c821ba in term 0.
I20260619 08:51:31.062870 24075 leader_election.cc:302] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1676542e561c4d6ea4d0b91225c821ba, 5e3deadcc2574864af2d9c40468635e3; no voters:
I20260619 08:51:31.063014 24220 raft_consensus.cc:2781] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:31.063083 24220 raft_consensus.cc:482] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:31.063115 24220 raft_consensus.cc:3037] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.064026 24220 raft_consensus.cc:504] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } }
I20260619 08:51:31.064163 24220 leader_election.cc:288] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [CANDIDATE]: Term 1 election: Requested vote from peers 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509), 588543e446414f85ae08087399584a6f (127.15.208.1:43335)
I20260619 08:51:31.064325 24003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6345f10f8940b88f7ff191eed3503a" candidate_uuid: "1676542e561c4d6ea4d0b91225c821ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e3deadcc2574864af2d9c40468635e3"
I20260619 08:51:31.064353 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6345f10f8940b88f7ff191eed3503a" candidate_uuid: "1676542e561c4d6ea4d0b91225c821ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "588543e446414f85ae08087399584a6f"
I20260619 08:51:31.064417 24003 raft_consensus.cc:3037] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.064419 23863 raft_consensus.cc:3037] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.065162 24003 raft_consensus.cc:2454] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1676542e561c4d6ea4d0b91225c821ba in term 1.
I20260619 08:51:31.065173 23863 raft_consensus.cc:2454] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1676542e561c4d6ea4d0b91225c821ba in term 1.
I20260619 08:51:31.065321 24075 leader_election.cc:302] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1676542e561c4d6ea4d0b91225c821ba, 5e3deadcc2574864af2d9c40468635e3; no voters:
I20260619 08:51:31.065425 24220 raft_consensus.cc:2781] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:31.065588 24220 raft_consensus.cc:686] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 1 LEADER]: Becoming Leader. State: Replica: 1676542e561c4d6ea4d0b91225c821ba, State: Running, Role: LEADER
I20260619 08:51:31.065693 24220 consensus_queue.cc:237] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } }
I20260619 08:51:31.066501 23706 catalog_manager.cc:5495] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3). New cstate: current_term: 1 leader_uuid: "1676542e561c4d6ea4d0b91225c821ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:31.085498 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.082096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.15.208.3:39075)
0619 08:51:31.082140 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.082142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.082151 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:31.082253 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.082254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.082275 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.082361 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:31.082527 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.083340 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.084534 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.084798 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.084878 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.085077 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:31.085209 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:31.085294 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.085375 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.085401 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:51:31.087271 24003 raft_consensus.cc:1261] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Refusing update from remote peer 1676542e561c4d6ea4d0b91225c821ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:31.087298 23863 raft_consensus.cc:1261] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Refusing update from remote peer 1676542e561c4d6ea4d0b91225c821ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:31.087595 24224 consensus_queue.cc:1035] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:31.087720 24220 consensus_queue.cc:1035] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:31.090739 24229 mvcc.cc:205] Tried to move back new op lower bound from 7298494837090836480 to 7298494837005799424. Current Snapshot: MvccSnapshot[applied={T|T < 7298494837090836480 or (T in {7298494837090836480})}]
I20260619 08:51:31.092284 23705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:31.092365 23705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:31.093156 23705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f: Sending DeleteTablet for 3 replicas of tablet 6a6345f10f8940b88f7ff191eed3503a
I20260619 08:51:31.093474 24123 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a6345f10f8940b88f7ff191eed3503a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:31 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:33828
I20260619 08:51:31.093549 23983 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a6345f10f8940b88f7ff191eed3503a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:31 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:39092
I20260619 08:51:31.093576 23843 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a6345f10f8940b88f7ff191eed3503a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:31 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53114
I20260619 08:51:31.093683 24236 tablet_replica.cc:330] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: stopping tablet replica
I20260619 08:51:31.093766 24236 raft_consensus.cc:2229] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:31.093896 24236 raft_consensus.cc:2258] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:31.093916 23705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:31.094084 23705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:31.094416 24236 ts_tablet_manager.cc:1905] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:31.094509 24237 tablet_replica.cc:330] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: stopping tablet replica
I20260619 08:51:31.094597 24237 raft_consensus.cc:2229] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:31.094692 24237 raft_consensus.cc:2258] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:31.095055 24238 tablet_replica.cc:330] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: stopping tablet replica
I20260619 08:51:31.095139 24238 raft_consensus.cc:2229] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:31.095237 24238 raft_consensus.cc:2258] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:31.095788 24123 tablet_service.cc:1467] Processing CreateTablet for tablet 43a3d3dc05274073a2c5bb7838fa7394 (DEFAULT_TABLE table=test-table [id=6ee8a651ab284802b8f072fd015f9651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:31.095904 24123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43a3d3dc05274073a2c5bb7838fa7394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.095973 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 43a3d3dc05274073a2c5bb7838fa7394 (DEFAULT_TABLE table=test-table [id=6ee8a651ab284802b8f072fd015f9651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:31.096078 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43a3d3dc05274073a2c5bb7838fa7394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.096153 24237 ts_tablet_manager.cc:1905] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:31.097514 24217 tablet_bootstrap.cc:492] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: Bootstrap starting.
I20260619 08:51:31.097577 23983 tablet_service.cc:1467] Processing CreateTablet for tablet 43a3d3dc05274073a2c5bb7838fa7394 (DEFAULT_TABLE table=test-table [id=6ee8a651ab284802b8f072fd015f9651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:31.097682 23983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43a3d3dc05274073a2c5bb7838fa7394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.097934 24217 tablet_bootstrap.cc:654] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.098279 24237 ts_tablet_manager.cc:1918] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:31.098393 24237 log.cc:1196] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/6a6345f10f8940b88f7ff191eed3503a
I20260619 08:51:31.098562 24236 ts_tablet_manager.cc:1918] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:31.098635 24236 log.cc:1196] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/6a6345f10f8940b88f7ff191eed3503a
I20260619 08:51:31.098881 24236 ts_tablet_manager.cc:1939] T 6a6345f10f8940b88f7ff191eed3503a P 1676542e561c4d6ea4d0b91225c821ba: Deleting consensus metadata
I20260619 08:51:31.099135 24237 ts_tablet_manager.cc:1939] T 6a6345f10f8940b88f7ff191eed3503a P 5e3deadcc2574864af2d9c40468635e3: Deleting consensus metadata
I20260619 08:51:31.099181 24215 tablet_bootstrap.cc:492] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: Bootstrap starting.
I20260619 08:51:31.099273 23693 catalog_manager.cc:4841] TS 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075): tablet 6a6345f10f8940b88f7ff191eed3503a (table test-table [id=dd85bce05ccd46c7a98e2072f34e0811]) successfully deleted
I20260619 08:51:31.099622 24215 tablet_bootstrap.cc:654] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.099731 24217 tablet_bootstrap.cc:492] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: No bootstrap required, opened a new log
I20260619 08:51:31.099773 24217 ts_tablet_manager.cc:1397] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 08:51:31.099920 24214 tablet_bootstrap.cc:492] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3: Bootstrap starting.
I20260619 08:51:31.099912 24217 raft_consensus.cc:348] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.099975 24217 raft_consensus.cc:374] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.099995 24217 raft_consensus.cc:729] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 588543e446414f85ae08087399584a6f, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.100039 24217 consensus_queue.cc:260] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.100122 24217 ts_tablet_manager.cc:1428] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:31.100178 24215 tablet_bootstrap.cc:492] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: No bootstrap required, opened a new log
I20260619 08:51:31.100216 24215 ts_tablet_manager.cc:1397] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:31.100306 24214 tablet_bootstrap.cc:654] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.100375 24215 raft_consensus.cc:348] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.100456 24215 raft_consensus.cc:374] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.100486 24215 raft_consensus.cc:729] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1676542e561c4d6ea4d0b91225c821ba, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.100538 24215 consensus_queue.cc:260] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.100642 24215 ts_tablet_manager.cc:1428] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:31.101060 23691 catalog_manager.cc:4841] TS 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509): tablet 6a6345f10f8940b88f7ff191eed3503a (table test-table [id=dd85bce05ccd46c7a98e2072f34e0811]) successfully deleted
I20260619 08:51:31.101511 24214 tablet_bootstrap.cc:492] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3: No bootstrap required, opened a new log
I20260619 08:51:31.101550 24214 ts_tablet_manager.cc:1397] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:31.101660 24214 raft_consensus.cc:348] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.101708 24214 raft_consensus.cc:374] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.101722 24214 raft_consensus.cc:729] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e3deadcc2574864af2d9c40468635e3, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.101774 24214 consensus_queue.cc:260] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.101863 24214 ts_tablet_manager.cc:1428] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:31.096082 24238 ts_tablet_manager.cc:1905] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:31.109244 24238 ts_tablet_manager.cc:1918] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:31.109328 24238 log.cc:1196] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/6a6345f10f8940b88f7ff191eed3503a
I20260619 08:51:31.109586 24238 ts_tablet_manager.cc:1939] T 6a6345f10f8940b88f7ff191eed3503a P 588543e446414f85ae08087399584a6f: Deleting consensus metadata
I20260619 08:51:31.110036 23691 catalog_manager.cc:4841] TS 588543e446414f85ae08087399584a6f (127.15.208.1:43335): tablet 6a6345f10f8940b88f7ff191eed3503a (table test-table [id=dd85bce05ccd46c7a98e2072f34e0811]) successfully deleted
I20260619 08:51:31.114240 24222 raft_consensus.cc:482] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:31.114313 24222 raft_consensus.cc:504] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.114621 24222 leader_election.cc:288] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509), 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075)
I20260619 08:51:31.117955 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.114893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:46479 (local address 127.15.208.3:39075)
0619 08:51:31.114941 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.114943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.114952 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:31.114978 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.114979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.114989 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.115060 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:31.115256 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.115793 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.117036 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.117608 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.117703 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.117712 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.117853 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.117888 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:31.117966 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.114787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:39075 (local address 127.15.208.1:46479)
0619 08:51:31.114849 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:31.114861 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:31.114907 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:31.115106 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:31.115107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:31.115109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:31.115173 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.115175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.115874 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.115875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.116950 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.116952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.117707 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.117707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.117778 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.117782 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:31.117838 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 08:51:31.117879 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 08:51:31.118317 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.115025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:33791 (local address 127.15.208.2:35509)
0619 08:51:31.115067 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.115069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.115151 (+ 82us) server_negotiation.cc:407] Connection header received
0619 08:51:31.115197 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.115198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.115205 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.115256 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:31.115440 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.116170 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.117459 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.117929 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.117993 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.118013 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.118210 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.118254 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:51:31.118324 24242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.114787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:35509 (local address 127.15.208.1:33791)
0619 08:51:31.115082 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:31.115091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:31.115141 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:31.115294 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:31.115295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:31.115297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:31.115369 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.115370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.116258 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.116259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.117378 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.117380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.118046 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.118047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.118132 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.118136 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:31.118195 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 08:51:31.118235 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":22,"thread_start_us":60,"threads_started":1}
I20260619 08:51:31.118350 24143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a3d3dc05274073a2c5bb7838fa7394" candidate_uuid: "588543e446414f85ae08087399584a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1676542e561c4d6ea4d0b91225c821ba" is_pre_election: true
I20260619 08:51:31.118480 24143 raft_consensus.cc:2454] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 588543e446414f85ae08087399584a6f in term 0.
I20260619 08:51:31.118490 24003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a3d3dc05274073a2c5bb7838fa7394" candidate_uuid: "588543e446414f85ae08087399584a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e3deadcc2574864af2d9c40468635e3" is_pre_election: true
I20260619 08:51:31.118571 24003 raft_consensus.cc:2454] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 588543e446414f85ae08087399584a6f in term 0.
I20260619 08:51:31.118697 23797 leader_election.cc:302] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1676542e561c4d6ea4d0b91225c821ba, 588543e446414f85ae08087399584a6f; no voters:
I20260619 08:51:31.118829 24222 raft_consensus.cc:2781] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:31.118870 24222 raft_consensus.cc:482] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:31.118891 24222 raft_consensus.cc:3037] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.119652 24222 raft_consensus.cc:504] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.119786 24222 leader_election.cc:288] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [CANDIDATE]: Term 1 election: Requested vote from peers 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509), 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075)
I20260619 08:51:31.119962 24003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a3d3dc05274073a2c5bb7838fa7394" candidate_uuid: "588543e446414f85ae08087399584a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e3deadcc2574864af2d9c40468635e3"
I20260619 08:51:31.120043 24003 raft_consensus.cc:3037] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.120707 24003 raft_consensus.cc:2454] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 588543e446414f85ae08087399584a6f in term 1.
I20260619 08:51:31.120817 24143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a3d3dc05274073a2c5bb7838fa7394" candidate_uuid: "588543e446414f85ae08087399584a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1676542e561c4d6ea4d0b91225c821ba"
I20260619 08:51:31.120898 24143 raft_consensus.cc:3037] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.120899 23795 leader_election.cc:302] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 588543e446414f85ae08087399584a6f, 5e3deadcc2574864af2d9c40468635e3; no voters:
I20260619 08:51:31.121711 24143 raft_consensus.cc:2454] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 588543e446414f85ae08087399584a6f in term 1.
I20260619 08:51:31.121822 24222 raft_consensus.cc:2781] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:31.121943 24222 raft_consensus.cc:686] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 1 LEADER]: Becoming Leader. State: Replica: 588543e446414f85ae08087399584a6f, State: Running, Role: LEADER
I20260619 08:51:31.122013 24222 consensus_queue.cc:237] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.122694 23706 catalog_manager.cc:5495] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 588543e446414f85ae08087399584a6f (127.15.208.1). New cstate: current_term: 1 leader_uuid: "588543e446414f85ae08087399584a6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260619 08:51:31.144300 23910 tablet.cc:2369] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 08:51:31 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859090, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.62@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:31.153561 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.145558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44612 (local address 127.15.208.62:43837)
0619 08:51:31.145627 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.145630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.145781 (+ 151us) server_negotiation.cc:407] Connection header received
0619 08:51:31.145856 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.145856 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.145865 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.145915 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:31.146490 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.147047 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.148133 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.148310 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.148362 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.150917 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:31.150918 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:31.150918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:31.150925 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:31.152318 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:31.152743 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:31.152744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:31.152744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:31.152803 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:31.152995 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:31.152996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:31.152996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:31.153107 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:31.153175 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.153334 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.153444 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:51:31.153625 24250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.145500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43837 (local address 127.0.0.1:44612)
0619 08:51:31.145702 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:31.145714 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:31.145793 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:31.145938 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:31.145939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:31.146355 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:31.146414 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.146417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.147124 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.147125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.148059 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.148062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.148383 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.148383 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.148444 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.149093 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:31.149098 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:31.150808 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:31.152417 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:31.152425 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:31.152428 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:31.152630 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:31.152868 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:31.152869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:31.152869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:31.152931 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:31.153178 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:31.153186 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:31.153229 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 08:51:31.153311 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:51:31.153505 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":91,"thread_start_us":94,"threads_started":1}
I20260619 08:51:31.155704 23706 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:44612:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 08:51:31.155902 23706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:31.157356 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 9d8f9028deef4f738e548e4cbe3f2016 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e309fb1fadb343b194f8e0b9f161faa9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:51:31.157383 24123 tablet_service.cc:1467] Processing CreateTablet for tablet 9d8f9028deef4f738e548e4cbe3f2016 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e309fb1fadb343b194f8e0b9f161faa9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:51:31.157362 23983 tablet_service.cc:1467] Processing CreateTablet for tablet 9d8f9028deef4f738e548e4cbe3f2016 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e309fb1fadb343b194f8e0b9f161faa9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:51:31.157514 24123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d8f9028deef4f738e548e4cbe3f2016. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.157519 23983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d8f9028deef4f738e548e4cbe3f2016. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.157524 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d8f9028deef4f738e548e4cbe3f2016. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.158968 24215 tablet_bootstrap.cc:492] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: Bootstrap starting.
I20260619 08:51:31.158998 24217 tablet_bootstrap.cc:492] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: Bootstrap starting.
I20260619 08:51:31.159298 24215 tablet_bootstrap.cc:654] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.159358 24217 tablet_bootstrap.cc:654] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.159696 24214 tablet_bootstrap.cc:492] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: Bootstrap starting.
I20260619 08:51:31.159915 24215 tablet_bootstrap.cc:492] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: No bootstrap required, opened a new log
I20260619 08:51:31.159966 24215 ts_tablet_manager.cc:1397] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:31.160120 24214 tablet_bootstrap.cc:654] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.160094 24215 raft_consensus.cc:348] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.160174 24215 raft_consensus.cc:374] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.160207 24215 raft_consensus.cc:729] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1676542e561c4d6ea4d0b91225c821ba, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.160256 24215 consensus_queue.cc:260] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.160359 24215 ts_tablet_manager.cc:1428] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:31.160470 24220 tablet_replica.cc:439] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } }
I20260619 08:51:31.160549 24220 tablet_replica.cc:442] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:31.160943 24214 tablet_bootstrap.cc:492] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: No bootstrap required, opened a new log
I20260619 08:51:31.160996 24214 ts_tablet_manager.cc:1397] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:31.161151 24217 tablet_bootstrap.cc:492] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: No bootstrap required, opened a new log
I20260619 08:51:31.161191 24217 ts_tablet_manager.cc:1397] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 08:51:31.161163 24214 raft_consensus.cc:348] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.161229 24214 raft_consensus.cc:374] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.161262 24214 raft_consensus.cc:729] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e3deadcc2574864af2d9c40468635e3, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.161299 24214 consensus_queue.cc:260] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.161319 24217 raft_consensus.cc:348] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.161382 24217 raft_consensus.cc:374] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.161394 24214 ts_tablet_manager.cc:1428] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:31.161410 24217 raft_consensus.cc:729] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 588543e446414f85ae08087399584a6f, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.161451 24217 consensus_queue.cc:260] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.161511 24221 tablet_replica.cc:439] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } }
I20260619 08:51:31.161574 24217 ts_tablet_manager.cc:1428] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:31.161593 24221 tablet_replica.cc:442] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:31.161620 24222 tablet_replica.cc:439] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } }
I20260619 08:51:31.161707 24222 tablet_replica.cc:442] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 08:51:31.195482 24190 tablet.cc:2369] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:51:31.421600 24221 raft_consensus.cc:482] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:31.421693 24221 raft_consensus.cc:504] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.422034 24221 leader_election.cc:288] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 588543e446414f85ae08087399584a6f (127.15.208.1:43335), 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075)
I20260619 08:51:31.425199 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.422279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:39075 (local address 127.15.208.2:47909)
0619 08:51:31.422408 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:31.422419 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:31.422492 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:31.422670 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:31.422671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:31.422673 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:31.422751 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.422753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.423439 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.423440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.424226 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.424228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.424964 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.424964 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.425036 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.425039 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:31.425093 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:51:31.425119 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260619 08:51:31.425218 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.422351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:47909 (local address 127.15.208.3:39075)
0619 08:51:31.422404 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.422406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.422491 (+ 85us) server_negotiation.cc:407] Connection header received
0619 08:51:31.422563 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.422564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.422581 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.422665 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:31.422830 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.423328 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.424366 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.424862 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.424959 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.424968 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.425109 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.425157 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 08:51:31.425217 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.422312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:40409 (local address 127.15.208.1:43335)
0619 08:51:31.422370 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.422373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.422443 (+ 70us) server_negotiation.cc:407] Connection header received
0619 08:51:31.422526 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.422527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.422536 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.422612 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:31.422778 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.423267 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.424312 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.424815 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.424914 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.424924 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.425091 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.425143 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:51:31.425205 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.422222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43335 (local address 127.15.208.2:40409)
0619 08:51:31.422367 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:31.422379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:31.422461 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:31.422616 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:31.422617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:31.422619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:31.422698 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.422700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.423371 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.423372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.424217 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.424220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.424915 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.424916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.424987 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.424992 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:51:31.425050 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:51:31.425097 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":49,"thread_start_us":71,"threads_started":1}
I20260619 08:51:31.425460 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d8f9028deef4f738e548e4cbe3f2016" candidate_uuid: "5e3deadcc2574864af2d9c40468635e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "588543e446414f85ae08087399584a6f" is_pre_election: true
I20260619 08:51:31.425547 23863 raft_consensus.cc:2454] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e3deadcc2574864af2d9c40468635e3 in term 0.
I20260619 08:51:31.425586 24143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d8f9028deef4f738e548e4cbe3f2016" candidate_uuid: "5e3deadcc2574864af2d9c40468635e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1676542e561c4d6ea4d0b91225c821ba" is_pre_election: true
I20260619 08:51:31.425710 24143 raft_consensus.cc:2454] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e3deadcc2574864af2d9c40468635e3 in term 0.
I20260619 08:51:31.425740 23935 leader_election.cc:302] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 588543e446414f85ae08087399584a6f, 5e3deadcc2574864af2d9c40468635e3; no voters:
I20260619 08:51:31.425863 24221 raft_consensus.cc:2781] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:31.425933 24221 raft_consensus.cc:482] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:31.425969 24221 raft_consensus.cc:3037] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.426693 24221 raft_consensus.cc:504] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.426856 24221 leader_election.cc:288] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [CANDIDATE]: Term 1 election: Requested vote from peers 588543e446414f85ae08087399584a6f (127.15.208.1:43335), 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075)
I20260619 08:51:31.427012 24143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d8f9028deef4f738e548e4cbe3f2016" candidate_uuid: "5e3deadcc2574864af2d9c40468635e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1676542e561c4d6ea4d0b91225c821ba"
I20260619 08:51:31.427034 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d8f9028deef4f738e548e4cbe3f2016" candidate_uuid: "5e3deadcc2574864af2d9c40468635e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "588543e446414f85ae08087399584a6f"
I20260619 08:51:31.427099 23863 raft_consensus.cc:3037] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.427109 24143 raft_consensus.cc:3037] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.427794 23863 raft_consensus.cc:2454] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e3deadcc2574864af2d9c40468635e3 in term 1.
I20260619 08:51:31.427803 24143 raft_consensus.cc:2454] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e3deadcc2574864af2d9c40468635e3 in term 1.
I20260619 08:51:31.427944 23935 leader_election.cc:302] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 588543e446414f85ae08087399584a6f, 5e3deadcc2574864af2d9c40468635e3; no voters:
I20260619 08:51:31.428043 24221 raft_consensus.cc:2781] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:31.428158 24221 raft_consensus.cc:686] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [term 1 LEADER]: Becoming Leader. State: Replica: 5e3deadcc2574864af2d9c40468635e3, State: Running, Role: LEADER
I20260619 08:51:31.428234 24221 consensus_queue.cc:237] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.428560 24256 tablet_replica.cc:439] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: TxnStatusTablet state changed. Reason: New leader 5e3deadcc2574864af2d9c40468635e3. Latest consensus state: current_term: 1 leader_uuid: "5e3deadcc2574864af2d9c40468635e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } }
I20260619 08:51:31.428634 24256 tablet_replica.cc:442] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:51:31.428817 24258 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 08:51:31.428954 24258 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 08:51:31.428987 23706 catalog_manager.cc:5495] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2). New cstate: current_term: 1 leader_uuid: "5e3deadcc2574864af2d9c40468635e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:31.448428 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.445493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58262 (local address 127.15.208.2:35509)
0619 08:51:31.445550 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.445552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.445615 (+ 63us) server_negotiation.cc:407] Connection header received
0619 08:51:31.445707 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.445708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.445727 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.445788 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:31.446127 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.446707 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.447596 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.447805 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.447879 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.448091 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:31.448195 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:31.448277 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.448333 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.448353 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:51:31.448462 24250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.445424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:35509 (local address 127.0.0.1:58262)
0619 08:51:31.445566 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:31.445578 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:31.445636 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:31.445843 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:31.445844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:31.445845 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:31.445897 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.445900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.446792 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.446793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.447515 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.447519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.447891 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.447892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.447973 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.447990 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:31.448262 (+ 272us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:31.448271 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 08:51:31.448328 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 08:51:31.448390 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260619 08:51:31.453356 23863 raft_consensus.cc:1261] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Refusing update from remote peer 5e3deadcc2574864af2d9c40468635e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:31.453426 24143 raft_consensus.cc:1261] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Refusing update from remote peer 5e3deadcc2574864af2d9c40468635e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:31.453634 24221 consensus_queue.cc:1035] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:31.453807 24221 consensus_queue.cc:1035] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:31.454267 24222 tablet_replica.cc:439] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: TxnStatusTablet state changed. Reason: New leader 5e3deadcc2574864af2d9c40468635e3. Latest consensus state: current_term: 1 leader_uuid: "5e3deadcc2574864af2d9c40468635e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } }
I20260619 08:51:31.454358 24222 tablet_replica.cc:442] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:31.454457 24220 tablet_replica.cc:439] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: TxnStatusTablet state changed. Reason: New leader 5e3deadcc2574864af2d9c40468635e3. Latest consensus state: current_term: 1 leader_uuid: "5e3deadcc2574864af2d9c40468635e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } }
I20260619 08:51:31.454566 24220 tablet_replica.cc:442] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:31.455403 24221 tablet_replica.cc:439] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e3deadcc2574864af2d9c40468635e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } }
I20260619 08:51:31.455452 24227 mvcc.cc:205] Tried to move back new op lower bound from 7298494838590906368 to 7298494838490693632. Current Snapshot: MvccSnapshot[applied={T|T < 7298494838590906368}]
I20260619 08:51:31.455495 24221 tablet_replica.cc:442] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:51:31.455538 24220 tablet_replica.cc:439] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e3deadcc2574864af2d9c40468635e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } }
I20260619 08:51:31.455552 24221 tablet_replica.cc:439] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e3deadcc2574864af2d9c40468635e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } }
I20260619 08:51:31.455602 24220 tablet_replica.cc:442] T 9d8f9028deef4f738e548e4cbe3f2016 P 1676542e561c4d6ea4d0b91225c821ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:31.455605 24221 tablet_replica.cc:442] T 9d8f9028deef4f738e548e4cbe3f2016 P 5e3deadcc2574864af2d9c40468635e3: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:51:31.455667 24228 mvcc.cc:205] Tried to move back new op lower bound from 7298494838590906368 to 7298494838490693632. Current Snapshot: MvccSnapshot[applied={T|T < 7298494838590906368}]
I20260619 08:51:31.456547 24222 tablet_replica.cc:439] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e3deadcc2574864af2d9c40468635e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } } }
I20260619 08:51:31.456630 24222 tablet_replica.cc:442] T 9d8f9028deef4f738e548e4cbe3f2016 P 588543e446414f85ae08087399584a6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:51:31.460642 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.457810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.15.208.1:43335)
0619 08:51:31.457866 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.457868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.457876 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:31.457911 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.457912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.457930 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.458009 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:31.458121 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.458887 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.459757 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.460013 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.460073 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.460252 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:31.460378 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:31.460459 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.460548 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.460571 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:51:31.465903 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.463065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:39039 (local address 127.15.208.2:35509)
0619 08:51:31.463139 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.463141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.463207 (+ 66us) server_negotiation.cc:407] Connection header received
0619 08:51:31.463269 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.463270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.463280 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.463334 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:31.463481 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.463960 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.465191 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.465395 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.465485 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.465659 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:31.465697 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:31.465756 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.465818 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.465835 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:51:31.465947 24267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.462988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:35509 (local address 127.15.208.1:39039)
0619 08:51:31.463121 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:31.463132 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:31.463209 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:31.463338 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:31.463339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:31.463340 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:31.463401 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.463402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.464042 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.464043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.465095 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.465097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.465487 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.465487 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.465556 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.465569 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:31.465757 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:31.465760 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:31.465813 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:51:31.465854 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260619 08:51:31.468302 24143 raft_consensus.cc:1261] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Refusing update from remote peer 588543e446414f85ae08087399584a6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:31.468402 24003 raft_consensus.cc:1261] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Refusing update from remote peer 588543e446414f85ae08087399584a6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:31.468580 24222 consensus_queue.cc:1035] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:31.468693 24243 consensus_queue.cc:1035] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:31.477504 24272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.474449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43335 (local address 127.15.208.2:39731)
0619 08:51:31.474598 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:31.474610 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:31.474658 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:31.474790 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:31.474791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:31.474792 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:31.474852 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.474853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.475696 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.475697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.476592 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:31.476595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.477053 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.477055 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:31.477124 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.477136 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:31.477322 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:31.477324 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:31.477376 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:51:31.477418 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260619 08:51:31.477495 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.474625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:39731 (local address 127.15.208.1:43335)
0619 08:51:31.474665 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.474666 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.474678 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:51:31.474705 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.474705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.474715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.474788 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:31.474898 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.475597 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.476745 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.476948 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.477039 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.477233 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:31.477263 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:31.477314 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.477380 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.477398 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:51:31.487596 23705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:31.487726 23705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:31.488382 23705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f: Sending DeleteTablet for 3 replicas of tablet 43a3d3dc05274073a2c5bb7838fa7394
I20260619 08:51:31.488648 23843 tablet_service.cc:1514] Processing DeleteTablet for tablet 43a3d3dc05274073a2c5bb7838fa7394 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:31 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53114
I20260619 08:51:31.488773 24238 tablet_replica.cc:330] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: stopping tablet replica
I20260619 08:51:31.488781 23983 tablet_service.cc:1514] Processing DeleteTablet for tablet 43a3d3dc05274073a2c5bb7838fa7394 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:31 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:39092
I20260619 08:51:31.488832 24238 raft_consensus.cc:2229] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:31.488862 24123 tablet_service.cc:1514] Processing DeleteTablet for tablet 43a3d3dc05274073a2c5bb7838fa7394 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:31 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:33828
I20260619 08:51:31.488914 24238 raft_consensus.cc:2258] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:31.488957 24236 tablet_replica.cc:330] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: stopping tablet replica
I20260619 08:51:31.489002 24237 tablet_replica.cc:330] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3: stopping tablet replica
I20260619 08:51:31.489034 24236 raft_consensus.cc:2229] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:31.489055 24237 raft_consensus.cc:2229] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:31.489079 24236 raft_consensus.cc:2258] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:31.489090 24237 raft_consensus.cc:2258] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:31.489503 24237 ts_tablet_manager.cc:1905] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:31.489571 24236 ts_tablet_manager.cc:1905] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:31.491262 24237 ts_tablet_manager.cc:1918] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:31.491256 24236 ts_tablet_manager.cc:1918] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:31.491343 24236 log.cc:1196] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/43a3d3dc05274073a2c5bb7838fa7394
I20260619 08:51:31.491341 24237 log.cc:1196] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/43a3d3dc05274073a2c5bb7838fa7394
I20260619 08:51:31.491587 24237 ts_tablet_manager.cc:1939] T 43a3d3dc05274073a2c5bb7838fa7394 P 5e3deadcc2574864af2d9c40468635e3: Deleting consensus metadata
I20260619 08:51:31.491600 24236 ts_tablet_manager.cc:1939] T 43a3d3dc05274073a2c5bb7838fa7394 P 1676542e561c4d6ea4d0b91225c821ba: Deleting consensus metadata
I20260619 08:51:31.491918 23691 catalog_manager.cc:4841] TS 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509): tablet 43a3d3dc05274073a2c5bb7838fa7394 (table test-table [id=6ee8a651ab284802b8f072fd015f9651]) successfully deleted
I20260619 08:51:31.491961 23693 catalog_manager.cc:4841] TS 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075): tablet 43a3d3dc05274073a2c5bb7838fa7394 (table test-table [id=6ee8a651ab284802b8f072fd015f9651]) successfully deleted
I20260619 08:51:31.492182 24238 ts_tablet_manager.cc:1905] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:31.493743 24238 ts_tablet_manager.cc:1918] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:31.493813 24238 log.cc:1196] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/43a3d3dc05274073a2c5bb7838fa7394
I20260619 08:51:31.494053 24238 ts_tablet_manager.cc:1939] T 43a3d3dc05274073a2c5bb7838fa7394 P 588543e446414f85ae08087399584a6f: Deleting consensus metadata
I20260619 08:51:31.495342 23691 catalog_manager.cc:4841] TS 588543e446414f85ae08087399584a6f (127.15.208.1:43335): tablet 43a3d3dc05274073a2c5bb7838fa7394 (table test-table [id=6ee8a651ab284802b8f072fd015f9651]) successfully deleted
Jun 19 08:51:31 dist-test-slave-p0h9 krb5kdc[23661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:51:31 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:31 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 08:51:31.522997 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:31.515804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.15.208.62:43837)
0619 08:51:31.515851 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.515854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.516042 (+ 188us) server_negotiation.cc:407] Connection header received
0619 08:51:31.516107 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.516108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.516120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.516186 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:31.516969 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.517774 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.518830 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.519149 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.519229 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.522917 (+ 3688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44714: BlockingRecv error: recv got EOF from 127.0.0.1:44714 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 08:51:31 dist-test-slave-p0h9 krb5kdc[23661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM
I20260619 08:51:31.532815 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.523806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44730 (local address 127.15.208.62:43837)
0619 08:51:31.523845 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.523847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.524105 (+ 258us) server_negotiation.cc:407] Connection header received
0619 08:51:31.524177 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.524178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.524187 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.524247 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:31.524843 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.525631 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.526686 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.526994 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.527059 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.529814 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:31.529815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:31.529816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:31.529823 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:31.531511 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:31.531966 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:31.531967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:31.531968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:31.532035 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:31.532253 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:31.532254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:31.532255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:31.532391 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:31.532527 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.532615 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.532694 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:51:31.534624 23705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:31.534795 23705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:31.536127 24123 tablet_service.cc:1467] Processing CreateTablet for tablet 991ccf7fe6fe40aa933b4cb8e506f4d8 (DEFAULT_TABLE table=test-table [id=8f945de4ce5a47f0b1322b3816371700]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:31.536199 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 991ccf7fe6fe40aa933b4cb8e506f4d8 (DEFAULT_TABLE table=test-table [id=8f945de4ce5a47f0b1322b3816371700]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:31.536265 24123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 991ccf7fe6fe40aa933b4cb8e506f4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.536334 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 991ccf7fe6fe40aa933b4cb8e506f4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.537829 24217 tablet_bootstrap.cc:492] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f: Bootstrap starting.
I20260619 08:51:31.537899 23983 tablet_service.cc:1467] Processing CreateTablet for tablet 991ccf7fe6fe40aa933b4cb8e506f4d8 (DEFAULT_TABLE table=test-table [id=8f945de4ce5a47f0b1322b3816371700]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:31.538028 23983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 991ccf7fe6fe40aa933b4cb8e506f4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.538267 24217 tablet_bootstrap.cc:654] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.539443 24214 tablet_bootstrap.cc:492] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3: Bootstrap starting.
I20260619 08:51:31.539465 24215 tablet_bootstrap.cc:492] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba: Bootstrap starting.
I20260619 08:51:31.539814 24215 tablet_bootstrap.cc:654] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.539860 24214 tablet_bootstrap.cc:654] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.540933 24215 tablet_bootstrap.cc:492] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba: No bootstrap required, opened a new log
I20260619 08:51:31.540997 24215 ts_tablet_manager.cc:1397] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:31.541096 24215 raft_consensus.cc:348] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.541169 24215 raft_consensus.cc:374] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.541189 24215 raft_consensus.cc:729] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1676542e561c4d6ea4d0b91225c821ba, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.541230 24215 consensus_queue.cc:260] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.541316 24215 ts_tablet_manager.cc:1428] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:31.548100 24214 tablet_bootstrap.cc:492] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3: No bootstrap required, opened a new log
I20260619 08:51:31.548158 24214 ts_tablet_manager.cc:1397] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260619 08:51:31.548305 24214 raft_consensus.cc:348] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.548378 24214 raft_consensus.cc:374] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.548400 24214 raft_consensus.cc:729] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e3deadcc2574864af2d9c40468635e3, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.548439 24214 consensus_queue.cc:260] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.548527 24214 ts_tablet_manager.cc:1428] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:31.549064 24217 tablet_bootstrap.cc:492] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f: No bootstrap required, opened a new log
I20260619 08:51:31.549137 24217 ts_tablet_manager.cc:1397] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20260619 08:51:31.549309 24217 raft_consensus.cc:348] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.549396 24217 raft_consensus.cc:374] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.549422 24217 raft_consensus.cc:729] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 588543e446414f85ae08087399584a6f, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.549479 24217 consensus_queue.cc:260] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.549578 24217 ts_tablet_manager.cc:1428] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:31.747320 24270 raft_consensus.cc:482] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:31.747447 24270 raft_consensus.cc:504] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.747690 24270 leader_election.cc:288] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509), 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075)
I20260619 08:51:31.747907 24143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991ccf7fe6fe40aa933b4cb8e506f4d8" candidate_uuid: "588543e446414f85ae08087399584a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1676542e561c4d6ea4d0b91225c821ba" is_pre_election: true
I20260619 08:51:31.747907 24003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991ccf7fe6fe40aa933b4cb8e506f4d8" candidate_uuid: "588543e446414f85ae08087399584a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e3deadcc2574864af2d9c40468635e3" is_pre_election: true
I20260619 08:51:31.748019 24003 raft_consensus.cc:2454] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 588543e446414f85ae08087399584a6f in term 0.
I20260619 08:51:31.748019 24143 raft_consensus.cc:2454] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 588543e446414f85ae08087399584a6f in term 0.
I20260619 08:51:31.748188 23795 leader_election.cc:302] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 588543e446414f85ae08087399584a6f, 5e3deadcc2574864af2d9c40468635e3; no voters:
I20260619 08:51:31.748293 24270 raft_consensus.cc:2781] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:31.748350 24270 raft_consensus.cc:482] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:31.748370 24270 raft_consensus.cc:3037] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.749323 24270 raft_consensus.cc:504] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.749476 24270 leader_election.cc:288] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [CANDIDATE]: Term 1 election: Requested vote from peers 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509), 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075)
I20260619 08:51:31.749629 24003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991ccf7fe6fe40aa933b4cb8e506f4d8" candidate_uuid: "588543e446414f85ae08087399584a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e3deadcc2574864af2d9c40468635e3"
I20260619 08:51:31.749675 24143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991ccf7fe6fe40aa933b4cb8e506f4d8" candidate_uuid: "588543e446414f85ae08087399584a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1676542e561c4d6ea4d0b91225c821ba"
I20260619 08:51:31.749727 24003 raft_consensus.cc:3037] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.749783 24143 raft_consensus.cc:3037] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:31.750437 24003 raft_consensus.cc:2454] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 588543e446414f85ae08087399584a6f in term 1.
I20260619 08:51:31.750540 24143 raft_consensus.cc:2454] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 588543e446414f85ae08087399584a6f in term 1.
I20260619 08:51:31.750654 23795 leader_election.cc:302] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 588543e446414f85ae08087399584a6f, 5e3deadcc2574864af2d9c40468635e3; no voters:
I20260619 08:51:31.750769 24270 raft_consensus.cc:2781] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:31.750844 24270 raft_consensus.cc:686] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 1 LEADER]: Becoming Leader. State: Replica: 588543e446414f85ae08087399584a6f, State: Running, Role: LEADER
I20260619 08:51:31.750900 24270 consensus_queue.cc:237] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.751524 23705 catalog_manager.cc:5495] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 588543e446414f85ae08087399584a6f (127.15.208.1). New cstate: current_term: 1 leader_uuid: "588543e446414f85ae08087399584a6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:31.769618 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:31.767270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41800 (local address 127.15.208.1:43335)
0619 08:51:31.767319 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:31.767321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:31.767330 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:31.767359 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:31.767360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:31.767369 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:31.767424 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:31.767578 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.768053 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.768904 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:31.769106 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:31.769184 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:31.769344 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:31.769385 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:31.769446 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:31.769508 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 08:51:31.769531 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:51:31.770499 24003 raft_consensus.cc:1261] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Refusing update from remote peer 588543e446414f85ae08087399584a6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:31.770502 24143 raft_consensus.cc:1261] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Refusing update from remote peer 588543e446414f85ae08087399584a6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:31.770730 24270 consensus_queue.cc:1035] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:31.770857 24243 consensus_queue.cc:1035] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:31.773841 23706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:31.773932 23706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:31.774487 23706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f: Sending DeleteTablet for 3 replicas of tablet 991ccf7fe6fe40aa933b4cb8e506f4d8
I20260619 08:51:31.774753 24123 tablet_service.cc:1514] Processing DeleteTablet for tablet 991ccf7fe6fe40aa933b4cb8e506f4d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:31 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:33828
I20260619 08:51:31.774760 23843 tablet_service.cc:1514] Processing DeleteTablet for tablet 991ccf7fe6fe40aa933b4cb8e506f4d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:31 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53114
I20260619 08:51:31.774813 23983 tablet_service.cc:1514] Processing DeleteTablet for tablet 991ccf7fe6fe40aa933b4cb8e506f4d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:31 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:39092
I20260619 08:51:31.774866 24236 tablet_replica.cc:330] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba: stopping tablet replica
I20260619 08:51:31.774901 24237 tablet_replica.cc:330] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3: stopping tablet replica
I20260619 08:51:31.774933 24236 raft_consensus.cc:2229] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:31.774955 24237 raft_consensus.cc:2229] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:31.774978 24236 raft_consensus.cc:2258] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:31.775002 24237 raft_consensus.cc:2258] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:31.775079 24238 tablet_replica.cc:330] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f: stopping tablet replica
I20260619 08:51:31.775141 24238 raft_consensus.cc:2229] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:31.775221 24238 raft_consensus.cc:2258] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:31.775388 24237 ts_tablet_manager.cc:1905] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:31.775498 24238 ts_tablet_manager.cc:1905] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:31.775657 24236 ts_tablet_manager.cc:1905] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:31.775631 23706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:31.775758 23706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:31.777011 24237 ts_tablet_manager.cc:1918] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:31.777016 24238 ts_tablet_manager.cc:1918] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:31.777076 24238 log.cc:1196] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/991ccf7fe6fe40aa933b4cb8e506f4d8
I20260619 08:51:31.777077 24237 log.cc:1196] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/991ccf7fe6fe40aa933b4cb8e506f4d8
I20260619 08:51:31.777166 24236 ts_tablet_manager.cc:1918] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:31.777220 24236 log.cc:1196] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/991ccf7fe6fe40aa933b4cb8e506f4d8
I20260619 08:51:31.777310 24238 ts_tablet_manager.cc:1939] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 588543e446414f85ae08087399584a6f: Deleting consensus metadata
I20260619 08:51:31.777354 24237 ts_tablet_manager.cc:1939] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 5e3deadcc2574864af2d9c40468635e3: Deleting consensus metadata
I20260619 08:51:31.777426 24236 ts_tablet_manager.cc:1939] T 991ccf7fe6fe40aa933b4cb8e506f4d8 P 1676542e561c4d6ea4d0b91225c821ba: Deleting consensus metadata
I20260619 08:51:31.777627 23691 catalog_manager.cc:4841] TS 588543e446414f85ae08087399584a6f (127.15.208.1:43335): tablet 991ccf7fe6fe40aa933b4cb8e506f4d8 (table test-table [id=8f945de4ce5a47f0b1322b3816371700]) successfully deleted
I20260619 08:51:31.777709 23693 catalog_manager.cc:4841] TS 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075): tablet 991ccf7fe6fe40aa933b4cb8e506f4d8 (table test-table [id=8f945de4ce5a47f0b1322b3816371700]) successfully deleted
I20260619 08:51:31.777794 23691 catalog_manager.cc:4841] TS 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509): tablet 991ccf7fe6fe40aa933b4cb8e506f4d8 (table test-table [id=8f945de4ce5a47f0b1322b3816371700]) successfully deleted
I20260619 08:51:31.778244 24123 tablet_service.cc:1467] Processing CreateTablet for tablet 3c64fffa0cc347849082cb28ee8bef12 (DEFAULT_TABLE table=test-table [id=e9cd6670f95f4e48aeb1ff38337abd97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:31.778221 23983 tablet_service.cc:1467] Processing CreateTablet for tablet 3c64fffa0cc347849082cb28ee8bef12 (DEFAULT_TABLE table=test-table [id=e9cd6670f95f4e48aeb1ff38337abd97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:31.778254 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 3c64fffa0cc347849082cb28ee8bef12 (DEFAULT_TABLE table=test-table [id=e9cd6670f95f4e48aeb1ff38337abd97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:31.778347 23983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c64fffa0cc347849082cb28ee8bef12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.778347 24123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c64fffa0cc347849082cb28ee8bef12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.778354 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c64fffa0cc347849082cb28ee8bef12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:31.779366 24217 tablet_bootstrap.cc:492] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f: Bootstrap starting.
I20260619 08:51:31.779479 24214 tablet_bootstrap.cc:492] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3: Bootstrap starting.
I20260619 08:51:31.779687 24217 tablet_bootstrap.cc:654] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.779747 24214 tablet_bootstrap.cc:654] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.779948 24215 tablet_bootstrap.cc:492] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba: Bootstrap starting.
I20260619 08:51:31.780200 24217 tablet_bootstrap.cc:492] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f: No bootstrap required, opened a new log
I20260619 08:51:31.780264 24217 ts_tablet_manager.cc:1397] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:31.780277 24214 tablet_bootstrap.cc:492] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3: No bootstrap required, opened a new log
I20260619 08:51:31.780289 24215 tablet_bootstrap.cc:654] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:31.780319 24214 ts_tablet_manager.cc:1397] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 08:51:31.780382 24217 raft_consensus.cc:348] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.780490 24217 raft_consensus.cc:374] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.780447 24214 raft_consensus.cc:348] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.780526 24217 raft_consensus.cc:729] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 588543e446414f85ae08087399584a6f, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.780527 24214 raft_consensus.cc:374] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.780563 24214 raft_consensus.cc:729] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e3deadcc2574864af2d9c40468635e3, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.780568 24217 consensus_queue.cc:260] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.780602 24214 consensus_queue.cc:260] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.780678 24217 ts_tablet_manager.cc:1428] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:31.780704 24214 ts_tablet_manager.cc:1428] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:31.781062 24215 tablet_bootstrap.cc:492] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba: No bootstrap required, opened a new log
I20260619 08:51:31.781116 24215 ts_tablet_manager.cc:1397] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:31.781220 24215 raft_consensus.cc:348] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.781291 24215 raft_consensus.cc:374] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:31.781313 24215 raft_consensus.cc:729] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1676542e561c4d6ea4d0b91225c821ba, State: Initialized, Role: FOLLOWER
I20260619 08:51:31.781358 24215 consensus_queue.cc:260] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:31.781438 24215 ts_tablet_manager.cc:1428] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:32.007145 24221 raft_consensus.cc:482] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:32.007256 24221 raft_consensus.cc:504] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:32.007442 24221 leader_election.cc:288] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 588543e446414f85ae08087399584a6f (127.15.208.1:43335), 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075)
I20260619 08:51:32.007674 24143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c64fffa0cc347849082cb28ee8bef12" candidate_uuid: "5e3deadcc2574864af2d9c40468635e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1676542e561c4d6ea4d0b91225c821ba" is_pre_election: true
I20260619 08:51:32.007663 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c64fffa0cc347849082cb28ee8bef12" candidate_uuid: "5e3deadcc2574864af2d9c40468635e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "588543e446414f85ae08087399584a6f" is_pre_election: true
I20260619 08:51:32.007771 24143 raft_consensus.cc:2454] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e3deadcc2574864af2d9c40468635e3 in term 0.
I20260619 08:51:32.007771 23863 raft_consensus.cc:2454] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e3deadcc2574864af2d9c40468635e3 in term 0.
I20260619 08:51:32.007910 23935 leader_election.cc:302] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 588543e446414f85ae08087399584a6f, 5e3deadcc2574864af2d9c40468635e3; no voters:
I20260619 08:51:32.008003 24221 raft_consensus.cc:2781] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:32.008062 24221 raft_consensus.cc:482] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:32.008077 24221 raft_consensus.cc:3037] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:32.008688 24221 raft_consensus.cc:504] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:32.008858 24221 leader_election.cc:288] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [CANDIDATE]: Term 1 election: Requested vote from peers 588543e446414f85ae08087399584a6f (127.15.208.1:43335), 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075)
I20260619 08:51:32.009009 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c64fffa0cc347849082cb28ee8bef12" candidate_uuid: "5e3deadcc2574864af2d9c40468635e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "588543e446414f85ae08087399584a6f"
I20260619 08:51:32.009114 23863 raft_consensus.cc:3037] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:32.009081 24143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c64fffa0cc347849082cb28ee8bef12" candidate_uuid: "5e3deadcc2574864af2d9c40468635e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1676542e561c4d6ea4d0b91225c821ba"
I20260619 08:51:32.009157 24143 raft_consensus.cc:3037] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:32.009888 24143 raft_consensus.cc:2454] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e3deadcc2574864af2d9c40468635e3 in term 1.
I20260619 08:51:32.009897 23863 raft_consensus.cc:2454] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e3deadcc2574864af2d9c40468635e3 in term 1.
I20260619 08:51:32.010061 23935 leader_election.cc:302] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 588543e446414f85ae08087399584a6f, 5e3deadcc2574864af2d9c40468635e3; no voters:
I20260619 08:51:32.010171 24221 raft_consensus.cc:2781] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:32.010279 24221 raft_consensus.cc:686] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 1 LEADER]: Becoming Leader. State: Replica: 5e3deadcc2574864af2d9c40468635e3, State: Running, Role: LEADER
I20260619 08:51:32.010353 24221 consensus_queue.cc:237] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } }
I20260619 08:51:32.010972 23705 catalog_manager.cc:5495] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2). New cstate: current_term: 1 leader_uuid: "5e3deadcc2574864af2d9c40468635e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e3deadcc2574864af2d9c40468635e3" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 35509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:32.069240 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:32.067063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58270 (local address 127.15.208.2:35509)
0619 08:51:32.067172 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:32.067173 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:32.067180 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:51:32.067198 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:32.067199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:32.067208 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:32.067261 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:32.067370 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:32.067832 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:32.068600 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:32.068845 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:32.068899 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:32.069024 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:32.069049 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:32.069080 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:32.069117 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 08:51:32.069133 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":70,"threads_started":1}
I20260619 08:51:32.073423 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:32.071371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:53337 (local address 127.15.208.2:35509)
0619 08:51:32.071422 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:32.071424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:32.071526 (+ 102us) server_negotiation.cc:407] Connection header received
0619 08:51:32.071580 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:32.071581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:32.071587 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:32.071624 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:32.071720 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:32.072146 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:32.072830 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:32.073015 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:32.073070 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:32.073210 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:32.073231 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:32.073269 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:32.073318 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 08:51:32.073337 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:51:32.073557 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:32.071316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:35509 (local address 127.15.208.2:53337)
0619 08:51:32.071465 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:32.071474 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:32.071529 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:32.071638 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:32.071638 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:32.071640 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:51:32.071672 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:32.071673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:32.072202 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:32.072203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:32.072729 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:32.072730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:32.073093 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:32.073094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:32.073138 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:32.073142 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:51:32.073282 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:51:32.073283 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:51:32.073318 (+ 35us) client_negotiation.cc:240] Negotiation successful
0619 08:51:32.073353 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260619 08:51:32.075376 24143 raft_consensus.cc:1261] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Refusing update from remote peer 5e3deadcc2574864af2d9c40468635e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:32.075389 23863 raft_consensus.cc:1261] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Refusing update from remote peer 5e3deadcc2574864af2d9c40468635e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:32.075616 24299 consensus_queue.cc:1035] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1676542e561c4d6ea4d0b91225c821ba" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:32.075708 24221 consensus_queue.cc:1035] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "588543e446414f85ae08087399584a6f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:32.085901 23705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:32.086004 23705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:32.086604 23705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ee8b79298cc4777b162bbd23fb5d01f: Sending DeleteTablet for 3 replicas of tablet 3c64fffa0cc347849082cb28ee8bef12
I20260619 08:51:32.086830 23843 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c64fffa0cc347849082cb28ee8bef12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:32 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53114
I20260619 08:51:32.086853 24123 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c64fffa0cc347849082cb28ee8bef12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:32 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:33828
I20260619 08:51:32.086928 23983 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c64fffa0cc347849082cb28ee8bef12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:32 UTC) from {username='oryx', principal='oryx/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:39092
I20260619 08:51:32.086947 24238 tablet_replica.cc:330] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f: stopping tablet replica
I20260619 08:51:32.087001 24238 raft_consensus.cc:2229] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:32.087005 24237 tablet_replica.cc:330] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3: stopping tablet replica
I20260619 08:51:32.087039 24238 raft_consensus.cc:2258] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:32.087045 24237 raft_consensus.cc:2229] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:32.087102 24236 tablet_replica.cc:330] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba: stopping tablet replica
I20260619 08:51:32.087123 24237 raft_consensus.cc:2258] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:32.087162 24236 raft_consensus.cc:2229] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:32.087199 24236 raft_consensus.cc:2258] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:32.087599 24238 ts_tablet_manager.cc:1905] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:32.087642 24236 ts_tablet_manager.cc:1905] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:32.088714 24237 ts_tablet_manager.cc:1905] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:32.088991 24238 ts_tablet_manager.cc:1918] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:32.089041 24238 log.cc:1196] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/3c64fffa0cc347849082cb28ee8bef12
I20260619 08:51:32.089128 24236 ts_tablet_manager.cc:1918] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:32.089178 24236 log.cc:1196] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/3c64fffa0cc347849082cb28ee8bef12
I20260619 08:51:32.089264 24238 ts_tablet_manager.cc:1939] T 3c64fffa0cc347849082cb28ee8bef12 P 588543e446414f85ae08087399584a6f: Deleting consensus metadata
I20260619 08:51:32.089402 24236 ts_tablet_manager.cc:1939] T 3c64fffa0cc347849082cb28ee8bef12 P 1676542e561c4d6ea4d0b91225c821ba: Deleting consensus metadata
I20260619 08:51:32.089654 23691 catalog_manager.cc:4841] TS 588543e446414f85ae08087399584a6f (127.15.208.1:43335): tablet 3c64fffa0cc347849082cb28ee8bef12 (table test-table [id=e9cd6670f95f4e48aeb1ff38337abd97]) successfully deleted
I20260619 08:51:32.089807 23693 catalog_manager.cc:4841] TS 1676542e561c4d6ea4d0b91225c821ba (127.15.208.3:39075): tablet 3c64fffa0cc347849082cb28ee8bef12 (table test-table [id=e9cd6670f95f4e48aeb1ff38337abd97]) successfully deleted
I20260619 08:51:32.090215 24237 ts_tablet_manager.cc:1918] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:51:32.090289 24237 log.cc:1196] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/3c64fffa0cc347849082cb28ee8bef12
I20260619 08:51:32.090483 24237 ts_tablet_manager.cc:1939] T 3c64fffa0cc347849082cb28ee8bef12 P 5e3deadcc2574864af2d9c40468635e3: Deleting consensus metadata
I20260619 08:51:32.090809 23691 catalog_manager.cc:4841] TS 5e3deadcc2574864af2d9c40468635e3 (127.15.208.2:35509): tablet 3c64fffa0cc347849082cb28ee8bef12 (table test-table [id=e9cd6670f95f4e48aeb1ff38337abd97]) successfully deleted
I20260619 08:51:32.091111 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 23778
I20260619 08:51:32.098198 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 23918
I20260619 08:51:32.103621 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 24058
I20260619 08:51:32.109002 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 23674
2026-06-19T08:51:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2935 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:32 dist-test-slave-p0h9 krb5kdc[24310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:32 dist-test-slave-p0h9 krb5kdc[24310](info): set up 2 sockets
Jun 19 08:51:32 dist-test-slave-p0h9 krb5kdc[24310](info): commencing operation
krb5kdc: starting...
W20260619 08:51:33.157989 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:33 dist-test-slave-p0h9 krb5kdc[24310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:33.273648 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:40853
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:39159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:40853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:33.354310 24323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:33.354523 24323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:33.354552 24323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:33.355727 24323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:33.355769 24323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:33.355782 24323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:33.355795 24323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:33.355808 24323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:33.357309 24323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:39159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:40853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:40853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:33.357568 24323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:33.357820 24323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:33.362747 24328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:33.362830 24329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:33.362756 24331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:33.362990 24323 server_base.cc:1034] running on GCE node
I20260619 08:51:33.363294 24323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:33.363673 24323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:33.364861 24323 hybrid_clock.cc:648] HybridClock initialized: now 1781859093364831 us; error 39 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T08:51:33Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1261 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:33 dist-test-slave-p0h9 krb5kdc[24336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:33 dist-test-slave-p0h9 krb5kdc[24336](info): set up 2 sockets
Jun 19 08:51:33 dist-test-slave-p0h9 krb5kdc[24336](info): commencing operation
krb5kdc: starting...
W20260619 08:51:34.413839 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:34 dist-test-slave-p0h9 krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:34.530957 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:44287
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:44287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:34.612214 24349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:34.612474 24349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:34.612526 24349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:34.613737 24349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:34.613803 24349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:34.613835 24349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:34.613864 24349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:34.613893 24349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:34.615357 24349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:44287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:44287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:34.615680 24349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:34.615911 24349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:34.620748 24355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:34.620764 24354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:34.620800 24349 server_base.cc:1034] running on GCE node
W20260619 08:51:34.620769 24357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:34.621259 24349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:34.621659 24349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:34.622839 24349 hybrid_clock.cc:648] HybridClock initialized: now 1781859094622822 us; error 49 us; skew 500 ppm
Jun 19 08:51:34 dist-test-slave-p0h9 krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859094, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:34.625378 24349 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:34.625816 24349 webserver.cc:460] Webserver started at http://127.15.208.62:36751/ using document root <none> and password file <none>
I20260619 08:51:34.626039 24349 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:34.626091 24349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:34.626241 24349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:34.627298 24349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "c578020790d147288fd99a4c2d8a099a"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "de62ab361bc5be8a40c060eeb819da66"
server_key_iv: "0290259199829303439a7a4b03176e13"
server_key_version: "encryptionkey@0"
I20260619 08:51:34.627656 24349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "c578020790d147288fd99a4c2d8a099a"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "de62ab361bc5be8a40c060eeb819da66"
server_key_iv: "0290259199829303439a7a4b03176e13"
server_key_version: "encryptionkey@0"
I20260619 08:51:34.629032 24349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:34.629812 24364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:34.629988 24349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:34.630105 24349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "c578020790d147288fd99a4c2d8a099a"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "de62ab361bc5be8a40c060eeb819da66"
server_key_iv: "0290259199829303439a7a4b03176e13"
server_key_version: "encryptionkey@0"
I20260619 08:51:34.630178 24349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:34.641908 24349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:34.643680 24349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:34.643838 24349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:34.647807 24349 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:44287
I20260619 08:51:34.647840 24416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:44287 every 8 connection(s)
I20260619 08:51:34.648197 24349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:34.649096 24417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:34.651332 24417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a: Bootstrap starting.
I20260619 08:51:34.651912 24417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:34.652205 24417 log.cc:824] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:34.652918 24417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a: No bootstrap required, opened a new log
I20260619 08:51:34.654425 24417 raft_consensus.cc:348] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } }
I20260619 08:51:34.654587 24417 raft_consensus.cc:374] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:34.654637 24417 raft_consensus.cc:729] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c578020790d147288fd99a4c2d8a099a, State: Initialized, Role: FOLLOWER
I20260619 08:51:34.654806 24417 consensus_queue.cc:260] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } }
I20260619 08:51:34.654886 24417 raft_consensus.cc:388] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:34.654929 24417 raft_consensus.cc:482] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:34.654978 24417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:34.655558 24417 raft_consensus.cc:504] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } }
I20260619 08:51:34.655643 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 24349
I20260619 08:51:34.655639 24417 leader_election.cc:302] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c578020790d147288fd99a4c2d8a099a; no voters:
I20260619 08:51:34.655773 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:34.655866 24417 leader_election.cc:288] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:34.655889 24420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:34.655999 16192 external_mini_cluster.cc:1428] Setting key f448811c31ef94a06aea4ac49233f04c
I20260619 08:51:34.656070 24420 raft_consensus.cc:686] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 1 LEADER]: Becoming Leader. State: Replica: c578020790d147288fd99a4c2d8a099a, State: Running, Role: LEADER
I20260619 08:51:34.656180 24417 sys_catalog.cc:564] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:34.656152 24420 consensus_queue.cc:237] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } }
I20260619 08:51:34.656973 24422 sys_catalog.cc:455] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c578020790d147288fd99a4c2d8a099a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } } }
I20260619 08:51:34.656996 24421 sys_catalog.cc:455] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c578020790d147288fd99a4c2d8a099a. Latest consensus state: current_term: 1 leader_uuid: "c578020790d147288fd99a4c2d8a099a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } } }
I20260619 08:51:34.657058 24422 sys_catalog.cc:458] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:34.657091 24421 sys_catalog.cc:458] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:34.657377 24427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:34.657984 24427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:34.660280 24427 catalog_manager.cc:1347] Generated new cluster ID: 399bae2079df47e39fa728070ae90894
I20260619 08:51:34.660333 24427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:51:34 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:34.667959 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:34.656481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33244 (local address 127.15.208.62:44287)
0619 08:51:34.656658 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:34.656669 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:34.656744 (+ 75us) server_negotiation.cc:407] Connection header received
0619 08:51:34.656858 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:34.656864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:34.656930 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:34.657836 (+ 906us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:34.657936 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:34.659082 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:34.660112 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:34.660692 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:34.660845 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:34.663633 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:34.663637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:34.663637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:34.663649 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:34.666271 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:34.666691 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:34.666694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:34.666695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:34.666817 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:34.667035 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:34.667036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:34.667036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:34.667358 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:34.667519 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:34.667602 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 08:51:34.667751 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260619 08:51:34.675701 24427 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:34.676240 24427 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:34.679155 24427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a: Generated new TSK 0
I20260619 08:51:34.679333 24427 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:51:34.711052 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44287
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:34.791674 24443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:34.791915 24443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:34.791951 24443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:34.793126 24443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:34.793196 24443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:34.793241 24443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:34.794682 24443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:34.794951 24443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:34.795183 24443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:34.800082 24449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:34.800179 24448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:34.800092 24451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:34.800274 24443 server_base.cc:1034] running on GCE node
I20260619 08:51:34.800493 24443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:34.800788 24443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:34.801940 24443 hybrid_clock.cc:648] HybridClock initialized: now 1781859094801929 us; error 40 us; skew 500 ppm
Jun 19 08:51:34 dist-test-slave-p0h9 krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859094, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:34.804234 24443 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:51:34.804703 24443 webserver.cc:460] Webserver started at http://127.15.208.1:35719/ using document root <none> and password file <none>
I20260619 08:51:34.804976 24443 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:34.805047 24443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:34.805194 24443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:34.806236 24443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "694441537002423fba2331466d6b7ba0"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "b01c544b93c26bc4ac3f98c4af9046b0"
server_key_iv: "70827310135f53e38b79cb6d4cfcc986"
server_key_version: "encryptionkey@0"
I20260619 08:51:34.806577 24443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "694441537002423fba2331466d6b7ba0"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "b01c544b93c26bc4ac3f98c4af9046b0"
server_key_iv: "70827310135f53e38b79cb6d4cfcc986"
server_key_version: "encryptionkey@0"
I20260619 08:51:34.807865 24443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:51:34.808560 24458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:34.808776 24443 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:34.808879 24443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "694441537002423fba2331466d6b7ba0"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "b01c544b93c26bc4ac3f98c4af9046b0"
server_key_iv: "70827310135f53e38b79cb6d4cfcc986"
server_key_version: "encryptionkey@0"
I20260619 08:51:34.808957 24443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:34.824334 24443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:34.826406 24443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:34.826577 24443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:34.826833 24443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:34.827148 24443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:34.827206 24443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:34.827250 24443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:34.827296 24443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:34.833449 24443 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:39563
I20260619 08:51:34.833495 24571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:39563 every 8 connection(s)
I20260619 08:51:34.833804 24443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:34.835906 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 24443
I20260619 08:51:34.836024 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:34.836217 16192 external_mini_cluster.cc:1428] Setting key 9a367e61b9e841ee8615b2ee85ba6c9a
Jun 19 08:51:34 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859094, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:34.843551 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:34.834523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:58645 (local address 127.15.208.62:44287)
0619 08:51:34.834610 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:34.834613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:34.834807 (+ 194us) server_negotiation.cc:407] Connection header received
0619 08:51:34.835258 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:34.835259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:34.835273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:34.835321 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:34.836294 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:34.837019 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:34.837985 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:34.838263 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:34.838319 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:34.840682 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:34.840684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:34.840684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:34.840692 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:34.842158 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:34.842578 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:34.842579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:34.842579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:34.842642 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:34.842885 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:34.842887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:34.842887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:34.843006 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:34.843077 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:34.843380 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 08:51:34.843465 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 08:51:34.843742 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:34.834486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:58645)
0619 08:51:34.834685 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:34.834720 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:34.835176 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:34.835382 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:34.835386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:34.835927 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:34.836195 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:34.836201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:34.837102 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:34.837103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:34.837884 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:34.837886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:34.838347 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:34.838348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:34.838528 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:34.839035 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:34.839045 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:34.840588 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:34.842258 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:34.842265 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:34.842267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:34.842482 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:34.842715 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:34.842715 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:34.842716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:34.842823 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:34.843078 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:34.843085 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:34.843248 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 08:51:34.843347 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 08:51:34.843494 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
I20260619 08:51:34.844189 24572 heartbeater.cc:347] Connected to a master server at 127.15.208.62:44287
I20260619 08:51:34.844298 24572 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:34.844465 24572 heartbeater.cc:511] Master 127.15.208.62:44287 requested a full tablet report, sending...
I20260619 08:51:34.844983 24381 ts_manager.cc:194] Registered new tserver with Master: 694441537002423fba2331466d6b7ba0 (127.15.208.1:39563)
I20260619 08:51:34.845598 24381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:58645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:51:34.862967 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44287
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:34.943193 24577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:34.943387 24577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:34.943408 24577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:34.944490 24577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:34.944523 24577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:34.944550 24577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:34.945919 24577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:34.946106 24577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:34.946271 24577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:34.951059 24585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:34.951074 24582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:34.951166 24583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:34.951256 24577 server_base.cc:1034] running on GCE node
I20260619 08:51:34.951550 24577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:34.951769 24577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:34.952927 24577 hybrid_clock.cc:648] HybridClock initialized: now 1781859094952916 us; error 32 us; skew 500 ppm
Jun 19 08:51:34 dist-test-slave-p0h9 krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859094, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:34.955302 24577 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:51:34.955794 24577 webserver.cc:460] Webserver started at http://127.15.208.2:36799/ using document root <none> and password file <none>
I20260619 08:51:34.956086 24577 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:34.956163 24577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:34.956319 24577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:34.957499 24577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "c619fbd066f34a3c8b40b49523a737c0"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "70c60d18faef02b70102bb1376edd5fe"
server_key_iv: "1f0a316594c0df76983dd3267de17370"
server_key_version: "encryptionkey@0"
I20260619 08:51:34.957868 24577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "c619fbd066f34a3c8b40b49523a737c0"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "70c60d18faef02b70102bb1376edd5fe"
server_key_iv: "1f0a316594c0df76983dd3267de17370"
server_key_version: "encryptionkey@0"
I20260619 08:51:34.959151 24577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:51:34.959872 24592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:34.960052 24577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:34.960171 24577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "c619fbd066f34a3c8b40b49523a737c0"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "70c60d18faef02b70102bb1376edd5fe"
server_key_iv: "1f0a316594c0df76983dd3267de17370"
server_key_version: "encryptionkey@0"
I20260619 08:51:34.960253 24577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:34.972111 24577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:34.973894 24577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:34.974078 24577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:34.974298 24577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:34.974591 24577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:34.974648 24577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:34.974678 24577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:34.974705 24577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:34.980527 24577 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:38839
I20260619 08:51:34.980576 24705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:38839 every 8 connection(s)
I20260619 08:51:34.980870 24577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:51:34 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859094, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:34.987437 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 24577
I20260619 08:51:34.987561 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:34.987778 16192 external_mini_cluster.cc:1428] Setting key 5aec2732d0c5289d2b2891395cc7ffd4
I20260619 08:51:34.990144 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:34.981572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:56297 (local address 127.15.208.62:44287)
0619 08:51:34.981658 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:34.981660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:34.981824 (+ 164us) server_negotiation.cc:407] Connection header received
0619 08:51:34.982345 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:34.982346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:34.982358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:34.982412 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:34.983313 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:34.983846 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:34.984844 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:34.985063 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:34.985151 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:34.987138 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:34.987140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:34.987140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:34.987147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:34.988623 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:34.989111 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:34.989112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:34.989112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:34.989166 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:34.989436 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:34.989436 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:34.989437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:34.989533 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:34.989582 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:34.989921 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 08:51:34.990019 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:51:34.990301 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:34.981537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:56297)
0619 08:51:34.981704 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:34.981742 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:34.982237 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:34.982475 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:34.982481 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:34.983022 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:34.983207 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:34.983211 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:34.983939 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:34.983940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:34.984749 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:34.984751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:34.985158 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:34.985159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:34.985285 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:34.985718 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:34.985725 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:34.987038 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:34.988702 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:34.988708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:34.988710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:34.989038 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:34.989231 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:34.989232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:34.989233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:34.989373 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:34.989596 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:34.989601 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:34.989762 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 08:51:34.989874 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:51:34.990054 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260619 08:51:34.990722 24706 heartbeater.cc:347] Connected to a master server at 127.15.208.62:44287
I20260619 08:51:34.990825 24706 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:34.990993 24706 heartbeater.cc:511] Master 127.15.208.62:44287 requested a full tablet report, sending...
I20260619 08:51:34.991362 24381 ts_manager.cc:194] Registered new tserver with Master: c619fbd066f34a3c8b40b49523a737c0 (127.15.208.2:38839)
I20260619 08:51:34.991734 24381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:56297
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:51:35.013495 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44287
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:35.093511 24711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:35.093705 24711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:35.093729 24711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:35.094874 24711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:35.094916 24711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:35.094969 24711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:35.096417 24711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:35.096665 24711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:35.096911 24711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:35.101716 24717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:35.101902 24716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:35.102015 24719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:35.102202 24711 server_base.cc:1034] running on GCE node
I20260619 08:51:35.102391 24711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:35.102641 24711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:35.103791 24711 hybrid_clock.cc:648] HybridClock initialized: now 1781859095103777 us; error 30 us; skew 500 ppm
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859095, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:35.106456 24711 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:51:35.106957 24711 webserver.cc:460] Webserver started at http://127.15.208.3:33787/ using document root <none> and password file <none>
I20260619 08:51:35.107201 24711 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:35.107250 24711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:35.107434 24711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:35.108521 24711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "7424b26fd8834422a3541e3504a479ae"
format_stamp: "Formatted at 2026-06-19 08:51:35 on dist-test-slave-p0h9"
server_key: "f0a420e6dc6d4d166b0f3176a341f743"
server_key_iv: "3b5a0b1bb651eed59460d692c334f0ef"
server_key_version: "encryptionkey@0"
I20260619 08:51:35.108939 24711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "7424b26fd8834422a3541e3504a479ae"
format_stamp: "Formatted at 2026-06-19 08:51:35 on dist-test-slave-p0h9"
server_key: "f0a420e6dc6d4d166b0f3176a341f743"
server_key_iv: "3b5a0b1bb651eed59460d692c334f0ef"
server_key_version: "encryptionkey@0"
I20260619 08:51:35.110320 24711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:35.111127 24726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.111363 24711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:35.111464 24711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "7424b26fd8834422a3541e3504a479ae"
format_stamp: "Formatted at 2026-06-19 08:51:35 on dist-test-slave-p0h9"
server_key: "f0a420e6dc6d4d166b0f3176a341f743"
server_key_iv: "3b5a0b1bb651eed59460d692c334f0ef"
server_key_version: "encryptionkey@0"
I20260619 08:51:35.111541 24711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:35.122968 24711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:35.125519 24711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:35.125661 24711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:35.125907 24711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:35.126307 24711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:35.126365 24711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.126401 24711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:35.126423 24711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.132975 24711 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:41967
I20260619 08:51:35.133046 24839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:41967 every 8 connection(s)
I20260619 08:51:35.133360 24711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:35.138185 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 24711
I20260619 08:51:35.138316 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:35.138520 16192 external_mini_cluster.cc:1428] Setting key da8e0accf647673c41251b5c896bdd69
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859095, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:35.142409 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.134103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:37379 (local address 127.15.208.62:44287)
0619 08:51:35.134182 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.134184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.134345 (+ 161us) server_negotiation.cc:407] Connection header received
0619 08:51:35.134795 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.134796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.134806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.134857 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.135724 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.136207 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.137250 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.137438 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.137492 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.139961 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:35.139962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:35.139962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:35.139968 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:35.141278 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:35.141640 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:35.141641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:35.141641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:35.141691 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:35.141895 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:35.141895 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:35.141896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:35.141996 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:35.142060 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:35.142282 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 08:51:35.142345 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:51:35.142609 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.134060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:37379)
0619 08:51:35.134223 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:35.134257 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:35.134716 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:35.134906 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:35.134909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:35.135451 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:35.135651 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.135656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.136285 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.136285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.137179 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.137181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.137538 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.137539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.137639 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.138114 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:35.138125 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:35.139883 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:35.141346 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:35.141351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:35.141353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:35.141574 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:35.141748 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:35.141749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:35.141749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:35.141842 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:35.142067 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:35.142070 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:35.142182 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 08:51:35.142262 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:51:35.142379 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260619 08:51:35.143087 24840 heartbeater.cc:347] Connected to a master server at 127.15.208.62:44287
I20260619 08:51:35.143198 24840 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:35.143391 24840 heartbeater.cc:511] Master 127.15.208.62:44287 requested a full tablet report, sending...
I20260619 08:51:35.143751 24381 ts_manager.cc:194] Registered new tserver with Master: 7424b26fd8834422a3541e3504a479ae (127.15.208.3:41967)
I20260619 08:51:35.144138 24381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:37379
I20260619 08:51:35.150259 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab.
Entry for principal oryx/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab.
WARNING: no policy specified for oryx/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab.
Entry for principal oryx/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab.
WARNING: no policy specified for oryx/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab.
Entry for principal oryx/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab.
I20260619 08:51:35.223451 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 24443
I20260619 08:51:35.227861 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 24577
I20260619 08:51:35.232023 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 24711
I20260619 08:51:35.236526 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 24349
I20260619 08:51:35.241524 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:44287
--webserver_interface=127.15.208.62
--webserver_port=36751
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:44287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:35.321106 24849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:35.321298 24849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:35.321318 24849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:35.322374 24849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:35.322404 24849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:35.322418 24849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:35.322430 24849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:35.322443 24849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:35.323788 24849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:44287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:44287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=36751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:35.323966 24849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:35.324131 24849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:35.328766 24854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:35.328701 24857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:35.328711 24855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:35.328969 24849 server_base.cc:1034] running on GCE node
I20260619 08:51:35.329201 24849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:35.329471 24849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:35.330633 24849 hybrid_clock.cc:648] HybridClock initialized: now 1781859095330628 us; error 47 us; skew 500 ppm
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859095, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:35.333161 24849 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:35.333520 24849 webserver.cc:460] Webserver started at http://127.15.208.62:36751/ using document root <none> and password file <none>
I20260619 08:51:35.333724 24849 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:35.333773 24849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:35.335012 24849 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:35.335650 24864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.335819 24849 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.335886 24849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "c578020790d147288fd99a4c2d8a099a"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "de62ab361bc5be8a40c060eeb819da66"
server_key_iv: "0290259199829303439a7a4b03176e13"
server_key_version: "encryptionkey@0"
I20260619 08:51:35.336186 24849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:35.351861 24849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:35.353773 24849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:35.353886 24849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:35.357625 24849 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:44287
I20260619 08:51:35.357683 24916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:44287 every 8 connection(s)
I20260619 08:51:35.358033 24849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:35.359479 24917 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:51:35.360141 24917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a: Bootstrap starting.
I20260619 08:51:35.366029 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 24849
I20260619 08:51:35.366333 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:39563
--local_ip_for_outbound_sockets=127.15.208.1
--tserver_master_addrs=127.15.208.62:44287
--webserver_port=35719
--webserver_interface=127.15.208.1
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.208.1
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:51:35.369115 24917 log.cc:824] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:35.370776 24917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:51:35.371068 24917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a: Bootstrap complete.
I20260619 08:51:35.373399 24917 raft_consensus.cc:348] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } }
I20260619 08:51:35.373646 24917 raft_consensus.cc:729] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c578020790d147288fd99a4c2d8a099a, State: Initialized, Role: FOLLOWER
I20260619 08:51:35.373827 24917 consensus_queue.cc:260] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } }
I20260619 08:51:35.373905 24917 raft_consensus.cc:388] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:35.373957 24917 raft_consensus.cc:482] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:35.374004 24917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:51:35.374963 24917 raft_consensus.cc:504] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } }
I20260619 08:51:35.375082 24917 leader_election.cc:302] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c578020790d147288fd99a4c2d8a099a; no voters:
I20260619 08:51:35.375275 24917 leader_election.cc:288] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:51:35.375319 24921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:51:35.375452 24921 raft_consensus.cc:686] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [term 2 LEADER]: Becoming Leader. State: Replica: c578020790d147288fd99a4c2d8a099a, State: Running, Role: LEADER
I20260619 08:51:35.375552 24917 sys_catalog.cc:564] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:35.375566 24921 consensus_queue.cc:237] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } }
I20260619 08:51:35.375943 24922 sys_catalog.cc:455] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c578020790d147288fd99a4c2d8a099a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } } }
I20260619 08:51:35.375948 24923 sys_catalog.cc:455] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c578020790d147288fd99a4c2d8a099a. Latest consensus state: current_term: 2 leader_uuid: "c578020790d147288fd99a4c2d8a099a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c578020790d147288fd99a4c2d8a099a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44287 } } }
I20260619 08:51:35.376139 24922 sys_catalog.cc:458] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:35.376156 24923 sys_catalog.cc:458] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:35.376781 24930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:35.377235 24930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:35.377386 24930 catalog_manager.cc:1259] Loaded cluster ID: 399bae2079df47e39fa728070ae90894
I20260619 08:51:35.377435 24930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:35.378191 24930 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:35.378372 24930 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c578020790d147288fd99a4c2d8a099a: Loaded TSK: 0
I20260619 08:51:35.378576 24930 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:51:35.453651 24919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:35.453881 24919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:35.453919 24919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:35.455070 24919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:35.455137 24919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:35.455183 24919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:35.456598 24919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.1.keytab
--principal=oryx/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:39563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=35719
--tserver_master_addrs=127.15.208.62:44287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:35.456912 24919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:35.457145 24919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:35.461925 24946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:35.461920 24943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:35.462245 24919 server_base.cc:1034] running on GCE node
W20260619 08:51:35.462308 24944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:35.462533 24919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:35.462772 24919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:35.463936 24919 hybrid_clock.cc:648] HybridClock initialized: now 1781859095463923 us; error 34 us; skew 500 ppm
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859095, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:35.466648 24919 init.cc:377] Logged in from keytab as oryx/127.15.208.1@KRBTEST.COM (short username oryx)
I20260619 08:51:35.467113 24919 webserver.cc:460] Webserver started at http://127.15.208.1:35719/ using document root <none> and password file <none>
I20260619 08:51:35.467352 24919 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:35.467435 24919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:35.468854 24919 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:35.469494 24953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.469669 24919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:35.469765 24919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "694441537002423fba2331466d6b7ba0"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "b01c544b93c26bc4ac3f98c4af9046b0"
server_key_iv: "70827310135f53e38b79cb6d4cfcc986"
server_key_version: "encryptionkey@0"
I20260619 08:51:35.470103 24919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:35.496825 24919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:35.499153 24919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:35.499315 24919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:35.499547 24919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:35.499900 24919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:35.499951 24919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.499994 24919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:35.500021 24919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.506065 24919 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:39563
I20260619 08:51:35.506125 25066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:39563 every 8 connection(s)
I20260619 08:51:35.506484 24919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:35.511897 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 24919
I20260619 08:51:35.512185 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:38839
--local_ip_for_outbound_sockets=127.15.208.2
--tserver_master_addrs=127.15.208.62:44287
--webserver_port=36799
--webserver_interface=127.15.208.2
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.208.2
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:35.514819 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.507165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:35179)
0619 08:51:35.507313 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:35.507363 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:35.507844 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:35.508204 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:35.508208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:35.508774 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:35.508957 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.508961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.509989 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.509989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.510751 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.510752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.511274 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.511275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.511374 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.511869 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:35.511877 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:35.514625 (+ 2748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
W20260619 08:51:35.514896 25070 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:35.507249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:35179 (local address 127.15.208.62:44287)
0619 08:51:35.507455 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.507462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.507493 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:51:35.507998 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.508003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.508042 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.508162 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.509053 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.509903 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.510823 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.511199 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.511274 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.514747 (+ 3473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:35179: BlockingRecv error: recv got EOF from 127.15.208.1:35179 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
W20260619 08:51:35.515040 25067 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:44287 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.208.62:44287: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:35.519673 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.515282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:59795)
0619 08:51:35.515324 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:35.515334 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:35.515386 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:35.515500 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:35.515501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:35.515628 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:35.515673 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.515674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.516294 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.516296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.516883 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.516885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.517415 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.517416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.517505 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.517935 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:35.517940 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:35.519587 (+ 1647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 08:51:35.519743 25070 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:35.515344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:59795 (local address 127.15.208.62:44287)
0619 08:51:35.515403 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.515405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.515414 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:35.515437 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.515438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.515449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.515501 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.515753 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.516216 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.516962 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.517168 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.517247 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.519674 (+ 2427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:59795: BlockingRecv error: recv got EOF from 127.15.208.1:59795 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:35.524086 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.520044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:40491)
0619 08:51:35.520092 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:35.520106 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:35.520151 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:35.520259 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:35.520260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:35.520383 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:35.520408 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.520409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.521102 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.521103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.521592 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.521593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.521954 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.521955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.522006 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.522416 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:35.522420 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:35.524006 (+ 1586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:51:35.524143 25070 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:35.520115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:40491 (local address 127.15.208.62:44287)
0619 08:51:35.520159 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.520161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.520171 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:35.520203 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.520204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.520213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.520258 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.520477 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.521023 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.521683 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.521872 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.521945 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.524080 (+ 2135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:40491: BlockingRecv error: recv got EOF from 127.15.208.1:40491 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:51:35.524372 25067 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:51:35.593793 25071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:35.593999 25071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:35.594022 25071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:35.595206 25071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:35.595248 25071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:35.595274 25071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:35.596715 25071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.2.keytab
--principal=oryx/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:38839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=36799
--tserver_master_addrs=127.15.208.62:44287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:35.596974 25071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:35.597205 25071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:35.601941 25076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:35.602020 25079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:35.601941 25077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:35.602276 25071 server_base.cc:1034] running on GCE node
I20260619 08:51:35.602478 25071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:35.602662 25071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:35.603794 25071 hybrid_clock.cc:648] HybridClock initialized: now 1781859095603760 us; error 52 us; skew 500 ppm
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859095, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:35.606307 25071 init.cc:377] Logged in from keytab as oryx/127.15.208.2@KRBTEST.COM (short username oryx)
I20260619 08:51:35.606755 25071 webserver.cc:460] Webserver started at http://127.15.208.2:36799/ using document root <none> and password file <none>
I20260619 08:51:35.606983 25071 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:35.607064 25071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:35.608362 25071 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:35.609038 25086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.609320 25071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:35.609428 25071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "c619fbd066f34a3c8b40b49523a737c0"
format_stamp: "Formatted at 2026-06-19 08:51:34 on dist-test-slave-p0h9"
server_key: "70c60d18faef02b70102bb1376edd5fe"
server_key_iv: "1f0a316594c0df76983dd3267de17370"
server_key_version: "encryptionkey@0"
I20260619 08:51:35.609764 25071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:35.618306 25071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:35.620136 25071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:35.620290 25071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:35.620513 25071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:35.620894 25071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:35.620949 25071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.620991 25071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:35.621017 25071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.626878 25071 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:38839
I20260619 08:51:35.626930 25199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:38839 every 8 connection(s)
I20260619 08:51:35.627261 25071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:51:35.627929 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 25071
I20260619 08:51:35.628280 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:41967
--local_ip_for_outbound_sockets=127.15.208.3
--tserver_master_addrs=127.15.208.62:44287
--webserver_port=33787
--webserver_interface=127.15.208.3
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.208.3
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:35.636592 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.627928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:34733)
0619 08:51:35.628132 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:35.628174 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:35.628849 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:35.629117 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:35.629121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:35.629733 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:35.629920 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.629925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.630929 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.630945 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.632396 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.632399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.632918 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.632920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.633088 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.633636 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:35.633646 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:35.636407 (+ 2761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
W20260619 08:51:35.636612 25070 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:35.627962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:34733 (local address 127.15.208.62:44287)
0619 08:51:35.628043 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.628046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.628253 (+ 207us) server_negotiation.cc:407] Connection header received
0619 08:51:35.628948 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.628950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.628967 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.629054 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.630021 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.630818 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.632495 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.632828 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.632890 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.636528 (+ 3638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:34733: BlockingRecv error: recv got EOF from 127.15.208.2:34733 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 08:51:35.637110 25200 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:44287 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.208.62:44287: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:35.642455 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.637378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:60355)
0619 08:51:35.637425 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:35.637434 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:35.637471 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:35.637600 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:35.637601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:35.637756 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:35.637814 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.637816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.638661 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.638662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.639611 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.639614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.639987 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.639988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.640061 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.640504 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:35.640509 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:35.642372 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:51:35.642495 25070 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:35.637423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:60355 (local address 127.15.208.62:44287)
0619 08:51:35.637466 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.637468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.637476 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:35.637531 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.637532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.637540 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.637595 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.637896 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.638522 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.639708 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.639904 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.639957 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.642452 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:60355: BlockingRecv error: recv got EOF from 127.15.208.2:60355 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:35.647480 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.642889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:54419)
0619 08:51:35.643012 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:35.643024 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:35.643079 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:35.643215 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:35.643216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:35.643361 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:35.643393 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.643394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.644103 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.644104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.644936 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.644938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.645267 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.645268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.645321 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.645692 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:35.645696 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:35.647401 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107}
W20260619 08:51:35.647517 25070 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:35.642967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:54419 (local address 127.15.208.62:44287)
0619 08:51:35.643018 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.643019 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.643064 (+ 45us) server_negotiation.cc:407] Connection header received
0619 08:51:35.643147 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.643148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.643155 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.643208 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.643587 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.644022 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.645015 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.645192 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.645245 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.647475 (+ 2230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:54419: BlockingRecv error: recv got EOF from 127.15.208.2:54419 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260619 08:51:35.647750 25200 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:51:35.716010 25203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:35.716236 25203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:35.716265 25203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:35.717432 25203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:35.717476 25203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:35.717504 25203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:35.719046 25203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/krb5kdc/oryx_127.15.208.3.keytab
--principal=oryx/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:41967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=33787
--tserver_master_addrs=127.15.208.62:44287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:35.719249 25203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:35.719441 25203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:35.724500 25208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:35.724587 25211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:35.724507 25209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:35.724671 25203 server_base.cc:1034] running on GCE node
I20260619 08:51:35.724938 25203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:35.725203 25203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:35.726378 25203 hybrid_clock.cc:648] HybridClock initialized: now 1781859095726350 us; error 40 us; skew 500 ppm
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859095, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:35.728881 25203 init.cc:377] Logged in from keytab as oryx/127.15.208.3@KRBTEST.COM (short username oryx)
I20260619 08:51:35.729332 25203 webserver.cc:460] Webserver started at http://127.15.208.3:33787/ using document root <none> and password file <none>
I20260619 08:51:35.729581 25203 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:35.729666 25203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:35.731026 25203 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:35.731639 25218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.731805 25203 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:35.731905 25203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "7424b26fd8834422a3541e3504a479ae"
format_stamp: "Formatted at 2026-06-19 08:51:35 on dist-test-slave-p0h9"
server_key: "f0a420e6dc6d4d166b0f3176a341f743"
server_key_iv: "3b5a0b1bb651eed59460d692c334f0ef"
server_key_version: "encryptionkey@0"
I20260619 08:51:35.732259 25203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:35.739051 25203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:35.741019 25203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:35.741186 25203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:35.741400 25203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:35.741701 25203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:35.741757 25203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.741798 25203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:35.741827 25203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:35.747668 25203 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:41967
I20260619 08:51:35.747712 25331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:41967 every 8 connection(s)
I20260619 08:51:35.748090 25203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:35.753815 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 25203
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 08:51:35.755429 25070 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:35.748843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:40857 (local address 127.15.208.62:44287)
0619 08:51:35.748927 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.748930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.749161 (+ 231us) server_negotiation.cc:407] Connection header received
0619 08:51:35.749651 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.749653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.749663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.749723 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.750624 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.751103 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.752060 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.752230 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.752280 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.755360 (+ 3080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:40857: BlockingRecv error: recv got EOF from 127.15.208.3:40857 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:51:35.755491 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.748837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:40857)
0619 08:51:35.749019 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:35.749054 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:35.749517 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:35.749775 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:35.749779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:35.750347 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:35.750536 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.750540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.751182 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.751182 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.751974 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.751976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.752301 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.752301 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.752414 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.753017 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:35.753025 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:35.755295 (+ 2270us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":111,"threads_started":1}
W20260619 08:51:35.755776 25332 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:44287 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.208.62:44287: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:35.759992 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.755976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:32869)
0619 08:51:35.756028 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:35.756037 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:35.756077 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:35.756198 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:35.756199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:35.756313 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:35.756351 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.756352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.757017 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.757018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.757487 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.757488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.757854 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.757855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.757926 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.758312 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:35.758315 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:35.759912 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 08:51:35.760042 25070 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:35.756031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:32869 (local address 127.15.208.62:44287)
0619 08:51:35.756070 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.756072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.756081 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:35.756135 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.756136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.756144 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.756186 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.756422 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.756938 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.757582 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.757772 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.757853 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.759979 (+ 2126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:32869: BlockingRecv error: recv got EOF from 127.15.208.3:32869 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:51:35.761855 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.754424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33248 (local address 127.15.208.62:44287)
0619 08:51:35.754546 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.754548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.754557 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:35.754614 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.754615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.754627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.754688 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.755284 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.756001 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.756843 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.757096 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.757172 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.758500 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:35.758504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:35.758504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:35.758515 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:35.760449 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:35.760835 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:35.760838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:35.760838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:35.760943 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:35.761151 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:35.761152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:35.761152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:35.761381 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:35.761535 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:35.761647 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 08:51:35.761758 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:35 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:35.764456 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:35.760310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:50873)
0619 08:51:35.760393 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:35.760404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:35.760451 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:35.760570 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:35.760570 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:35.760691 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:35.760712 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.760713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.761338 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.761339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.761866 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:35.761868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.762231 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.762232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:35.762274 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.762688 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:35.762693 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:35.764389 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260619 08:51:35.764495 25070 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:35.760353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:50873 (local address 127.15.208.62:44287)
0619 08:51:35.760408 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:35.760410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:35.760438 (+ 28us) server_negotiation.cc:407] Connection header received
0619 08:51:35.760513 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:35.760513 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:35.760521 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:35.760568 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:35.760815 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.761258 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.761926 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:35.762146 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:35.762205 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:35.764459 (+ 2254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:50873: BlockingRecv error: recv got EOF from 127.15.208.3:50873 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 08:51:35.764667 25332 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 08:51:36 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:36 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:36.529259 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:36.524810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:57015)
0619 08:51:36.524962 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:36.524973 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:36.525025 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:36.525163 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:36.525163 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:36.525317 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:36.525364 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:36.525366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.526012 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.526013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:36.526560 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:36.526562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.526898 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.526899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:36.526937 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:36.527357 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:36.527361 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:36.529191 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
W20260619 08:51:36.529356 25338 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:36.524901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:57015 (local address 127.15.208.62:44287)
0619 08:51:36.525020 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:36.525022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:36.525031 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:36.525077 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:36.525078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:36.525087 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:36.525145 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:36.525434 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.525944 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.526638 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.526833 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.526887 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:36.529282 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:57015: BlockingRecv error: recv got EOF from 127.15.208.1:57015 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
Jun 19 08:51:36 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:36 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:36.653117 25339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:36.648180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:51785)
0619 08:51:36.648354 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:36.648368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:36.648448 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:36.648608 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:36.648609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:36.648866 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:36.648913 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:36.648915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.649591 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.649593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:36.650143 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:36.650145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.650529 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.650529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:36.650599 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:36.651058 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:36.651062 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:36.653014 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
W20260619 08:51:36.653192 25338 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:36.648283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:51785 (local address 127.15.208.62:44287)
0619 08:51:36.648375 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:36.648377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:36.648427 (+ 50us) server_negotiation.cc:407] Connection header received
0619 08:51:36.648523 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:36.648523 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:36.648535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:36.648592 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:36.648997 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.649499 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.650240 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.650441 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.650528 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:36.653114 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:51785: BlockingRecv error: recv got EOF from 127.15.208.2:51785 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 08:51:36 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:36 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:36.770139 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:36.765096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:35619)
0619 08:51:36.765307 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:36.765319 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:36.765411 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:36.765598 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:36.765599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:36.765785 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:36.765831 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:36.765833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.766509 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.766511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:36.767072 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:36.767074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.767470 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.767471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:36.767526 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:36.767982 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:36.767987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:36.770037 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":102,"threads_started":1}
W20260619 08:51:36.770201 25338 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:36.765215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:35619 (local address 127.15.208.62:44287)
0619 08:51:36.765291 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:36.765293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:36.765390 (+ 97us) server_negotiation.cc:407] Connection header received
0619 08:51:36.765474 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:36.765475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:36.765489 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:36.765588 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:36.765921 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.766420 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.767173 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:36.767364 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:36.767472 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:36.770154 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:35619: BlockingRecv error: recv got EOF from 127.15.208.3:35619 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 08:51:37 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:37 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:37.534274 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:37.529730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:59455)
0619 08:51:37.529878 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:37.529889 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:37.529946 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:37.530107 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:37.530108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:37.530258 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:37.530305 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:37.530307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.530923 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.530923 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:37.531447 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:37.531448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.531781 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.531782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:37.531824 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:37.532252 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:37.532257 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:37.534216 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
W20260619 08:51:37.534435 25342 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:37.529812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:59455 (local address 127.15.208.62:44287)
0619 08:51:37.529910 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:37.529918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:37.529928 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:37.530022 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:37.530023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:37.530033 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:37.530091 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:37.530372 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.530859 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.531528 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.531715 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.531767 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:37.534334 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:59455: BlockingRecv error: recv got EOF from 127.15.208.1:59455 (error 108)
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":48,"threads_started":1}
Jun 19 08:51:37 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:37 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:37.657826 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:37.653659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:52473)
0619 08:51:37.653793 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:37.653804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:37.653860 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:37.654011 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:37.654012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:37.654172 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:37.654219 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:37.654221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.654857 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.654858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:37.655362 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:37.655363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.655683 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.655684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:37.655722 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:37.656146 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:37.656150 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:37.657772 (+ 1622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
W20260619 08:51:37.657933 25342 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:37.653741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:52473 (local address 127.15.208.62:44287)
0619 08:51:37.653836 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:37.653838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:37.653851 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:51:37.653921 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:37.653922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:37.653932 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:37.653994 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:37.654282 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.654791 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.655433 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.655622 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.655672 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:37.657864 (+ 2192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:52473: BlockingRecv error: recv got EOF from 127.15.208.2:52473 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 08:51:37 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:37 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:37.775301 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:37.770726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:57619)
0619 08:51:37.770935 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:37.770950 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:37.771033 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:37.771190 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:37.771191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:37.771352 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:37.771399 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:37.771401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.772022 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.772023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:37.772547 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:37.772548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.773071 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.773072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:37.773117 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:37.773539 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:37.773543 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:37.775243 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":152,"threads_started":1}
W20260619 08:51:37.775545 25342 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:37.770868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:57619 (local address 127.15.208.62:44287)
0619 08:51:37.770920 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:37.770927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:37.771030 (+ 103us) server_negotiation.cc:407] Connection header received
0619 08:51:37.771083 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:37.771084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:37.771094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:37.771147 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:37.771469 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.771955 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.772639 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:37.772998 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:37.773068 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:37.775481 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:57619: BlockingRecv error: recv got EOF from 127.15.208.3:57619 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 08:51:38 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:38 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:38.541417 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:38.534829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:49451)
0619 08:51:38.534988 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:38.535007 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:38.535085 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:38.535284 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:38.535285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:38.535588 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:38.535670 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:38.535672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.536797 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.536798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:38.537833 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:38.537836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.538430 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.538431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:38.538488 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:38.539151 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:38.539157 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:38.541316 (+ 2159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
W20260619 08:51:38.541457 25346 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:38.534905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:49451 (local address 127.15.208.62:44287)
0619 08:51:38.535065 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:38.535067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:38.535079 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:51:38.535190 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:38.535192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:38.535206 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:38.535279 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:38.535771 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.536682 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.537939 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.538331 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.538412 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:38.541404 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:49451: BlockingRecv error: recv got EOF from 127.15.208.1:49451 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
Jun 19 08:51:38 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:38 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:38.662998 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:38.658274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:50661)
0619 08:51:38.658492 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:38.658504 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:38.658601 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:38.658730 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:38.658731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:38.658915 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:38.658961 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:38.658963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.659642 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.659643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:38.660179 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:38.660180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.660572 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.660572 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:38.660627 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:38.661091 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:38.661096 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:38.662940 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":113,"threads_started":1}
W20260619 08:51:38.663144 25346 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:38.658346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:50661 (local address 127.15.208.62:44287)
0619 08:51:38.658490 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:38.658492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:38.658594 (+ 102us) server_negotiation.cc:407] Connection header received
0619 08:51:38.658661 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:38.658662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:38.658674 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:38.658726 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:38.659054 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.659553 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.660264 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.660475 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.660573 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:38.663063 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:50661: BlockingRecv error: recv got EOF from 127.15.208.2:50661 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Jun 19 08:51:38 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:38 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:38.780579 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:38.775910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:43457)
0619 08:51:38.776119 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:38.776134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:38.776210 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:38.776384 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:38.776385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:38.776564 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:38.776613 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:38.776615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.777361 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.777362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:38.777879 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:38.777880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.778266 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.778267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:38.778331 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:38.778782 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:38.778786 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:38.780494 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":90,"threads_started":1}
W20260619 08:51:38.780682 25346 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:38.775979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:43457 (local address 127.15.208.62:44287)
0619 08:51:38.776077 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:38.776080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:38.776216 (+ 136us) server_negotiation.cc:407] Connection header received
0619 08:51:38.776278 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:38.776279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:38.776293 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:38.776356 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:38.776696 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.777266 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.777965 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:38.778177 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:38.778265 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:38.780607 (+ 2342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:43457: BlockingRecv error: recv got EOF from 127.15.208.3:43457 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Jun 19 08:51:39 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:39 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:39.550513 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:39.542183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:52121)
0619 08:51:39.542335 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:39.542345 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:39.542397 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:39.542540 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:39.542541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:39.542700 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:39.542749 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:39.542750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.543375 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.543376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:39.543903 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:39.543904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.544215 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.544216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:39.544252 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:39.548618 (+ 4366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:39.548623 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:39.550444 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
W20260619 08:51:39.550674 25350 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:39.542249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:52121 (local address 127.15.208.62:44287)
0619 08:51:39.542384 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:39.542386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:39.542395 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:39.542458 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:39.542459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:39.542469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:39.542527 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:39.542809 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.543310 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.543970 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.544154 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.544202 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:39.550581 (+ 6379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:52121: BlockingRecv error: recv got EOF from 127.15.208.1:52121 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
Jun 19 08:51:39 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:39 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:39.667953 25351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:39.663509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:45003)
0619 08:51:39.663654 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:39.663665 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:39.663742 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:39.663876 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:39.663877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:39.664030 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:39.664078 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:39.664080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.664941 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.664943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:39.665474 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:39.665476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.665806 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.665807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:39.665844 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:39.666252 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:39.666256 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:39.667896 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
W20260619 08:51:39.668047 25350 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:39.663576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:45003 (local address 127.15.208.62:44287)
0619 08:51:39.663656 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:39.663658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:39.663726 (+ 68us) server_negotiation.cc:407] Connection header received
0619 08:51:39.663788 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:39.663789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:39.663799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:39.663859 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:39.664139 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.664853 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.665563 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.665741 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.665794 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:39.667984 (+ 2190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:45003: BlockingRecv error: recv got EOF from 127.15.208.2:45003 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 08:51:39 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:39 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:39.785975 25352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:39.781114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:45091)
0619 08:51:39.781274 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:39.781285 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:39.781376 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:39.781586 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:39.781587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:39.781763 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:39.781808 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:39.781809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.782471 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.782472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:39.782991 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:39.782993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.783365 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.783367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:39.783428 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:39.783944 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:39.783953 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:39.785880 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
W20260619 08:51:39.786017 25350 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:39.781192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:45091 (local address 127.15.208.62:44287)
0619 08:51:39.781263 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:39.781266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:39.781357 (+ 91us) server_negotiation.cc:407] Connection header received
0619 08:51:39.781452 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:39.781453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:39.781466 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:39.781576 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:39.781885 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.782384 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.783083 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:39.783289 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:39.783368 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:39.785957 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:45091: BlockingRecv error: recv got EOF from 127.15.208.3:45091 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 08:51:40 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:40 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:40.555740 25353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:40.551034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:45153)
0619 08:51:40.551171 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:40.551184 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:40.551256 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:40.551397 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:40.551397 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:40.551584 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:40.551634 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:40.551635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.552394 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.552395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:40.553109 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:40.553112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.553436 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.553436 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:40.553473 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:40.553989 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:40.553993 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:40.555682 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
W20260619 08:51:40.555850 25354 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:40.551132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:45153 (local address 127.15.208.62:44287)
0619 08:51:40.551265 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:40.551267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:40.551276 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:40.551305 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:40.551305 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:40.551316 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:40.551373 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:40.551701 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.552284 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.553177 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.553368 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.553422 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:40.555772 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:45153: BlockingRecv error: recv got EOF from 127.15.208.1:45153 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
Jun 19 08:51:40 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:40 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:40.673635 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:40.668588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:38775)
0619 08:51:40.668796 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:40.668809 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:40.668884 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:40.669056 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:40.669057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:40.669243 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:40.669290 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:40.669292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.669983 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.669984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:40.670498 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:40.670500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.670867 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.670868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:40.670919 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:40.671414 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:40.671419 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:40.673539 (+ 2120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":109,"threads_started":1}
W20260619 08:51:40.673684 25354 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:40.668765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:38775 (local address 127.15.208.62:44287)
0619 08:51:40.668824 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:40.668827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:40.668863 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:51:40.668950 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:40.668951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:40.668969 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:40.669049 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:40.669374 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.669882 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.670588 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.670785 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.670870 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:40.673620 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:38775: BlockingRecv error: recv got EOF from 127.15.208.2:38775 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 08:51:40 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:40 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:40.791492 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:40.786526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:51817)
0619 08:51:40.786727 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:40.786740 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:40.786828 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:40.787023 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:40.787024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:40.787204 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:40.787249 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:40.787251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.787964 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.787965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:40.788582 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:40.788584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.789044 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.789045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:40.789110 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:40.789632 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:40.789637 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:40.791398 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":118,"threads_started":1}
W20260619 08:51:40.791532 25354 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:40.786630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:51817 (local address 127.15.208.62:44287)
0619 08:51:40.786712 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:40.786715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:40.786812 (+ 97us) server_negotiation.cc:407] Connection header received
0619 08:51:40.786914 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:40.786915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:40.786949 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:40.787015 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:40.787346 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.787857 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.788683 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:40.788972 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:40.789046 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:40.791474 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:51817: BlockingRecv error: recv got EOF from 127.15.208.3:51817 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 08:51:41 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:41 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:41.560523 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:41.556241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:32863)
0619 08:51:41.556378 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:41.556395 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:41.556452 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:41.556603 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:41.556604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:41.556783 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:41.556833 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:41.556834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.557467 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.557468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:41.557979 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:41.557981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.558294 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.558295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:41.558333 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:41.558793 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:41.558797 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:41.560465 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
W20260619 08:51:41.560655 25358 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:41.556338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:32863 (local address 127.15.208.62:44287)
0619 08:51:41.556463 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:41.556465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:41.556498 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:51:41.556521 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:41.556522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:41.556532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:41.556584 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:41.556898 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.557400 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.558051 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.558233 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.558283 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:41.560568 (+ 2285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:32863: BlockingRecv error: recv got EOF from 127.15.208.1:32863 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
Jun 19 08:51:41 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:41 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:41.678908 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:41.674061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:56043)
0619 08:51:41.674174 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:41.674186 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:41.674257 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:41.674394 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:41.674394 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:41.674592 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:41.674641 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:41.674642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.675329 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.675330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:41.675959 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:41.675961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.676388 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.676389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:41.676446 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:41.677047 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:41.677052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:41.678854 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
W20260619 08:51:41.679006 25358 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:41.674132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:56043 (local address 127.15.208.62:44287)
0619 08:51:41.674204 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:41.674206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:41.674257 (+ 51us) server_negotiation.cc:407] Connection header received
0619 08:51:41.674316 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:41.674317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:41.674328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:41.674378 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:41.674705 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.675262 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.676115 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.676298 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.676355 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:41.678941 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:56043: BlockingRecv error: recv got EOF from 127.15.208.2:56043 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 19 08:51:41 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:41 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:41.796489 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:41.792042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:55441)
0619 08:51:41.792206 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:41.792216 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:41.792273 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:41.792378 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:41.792379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:41.792572 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:41.792617 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:41.792619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.793272 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.793273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:41.793793 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:41.793794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.794093 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.794094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:41.794131 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:41.794647 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:41.794652 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:41.796423 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
W20260619 08:51:41.796639 25358 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:41.792103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:55441 (local address 127.15.208.62:44287)
0619 08:51:41.792180 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:41.792182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:41.792274 (+ 92us) server_negotiation.cc:407] Connection header received
0619 08:51:41.792323 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:41.792324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:41.792336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:41.792373 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:41.792677 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.793202 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.793863 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:41.794033 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:41.794081 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:41.796520 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:55441: BlockingRecv error: recv got EOF from 127.15.208.3:55441 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 19 08:51:42 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:42 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:42.566219 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:42.561145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:51085)
0619 08:51:42.561315 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:42.561333 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:42.561405 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:42.561700 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:42.561701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:42.561942 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:42.562008 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:42.562011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.562753 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.562754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:42.563318 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:42.563320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.563683 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.563684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:42.563734 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:42.564190 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:42.564194 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:42.566158 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":62,"threads_started":1}
W20260619 08:51:42.566331 25362 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:42.561336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:51085 (local address 127.15.208.62:44287)
0619 08:51:42.561492 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:42.561494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:42.561521 (+ 27us) server_negotiation.cc:407] Connection header received
0619 08:51:42.561545 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:42.561545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:42.561556 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:42.561649 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:42.562104 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.562669 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.563396 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.563598 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.563650 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:42.566254 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:51085: BlockingRecv error: recv got EOF from 127.15.208.1:51085 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
Jun 19 08:51:42 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:42 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:42.684154 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:42.679413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:53073)
0619 08:51:42.679571 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:42.679581 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:42.679643 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:42.679766 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:42.679766 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:42.679924 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:42.679972 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:42.679974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.680901 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.680902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:42.681472 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:42.681474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.681786 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.681787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:42.681824 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:42.682254 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:42.682258 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:42.684070 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
W20260619 08:51:42.684278 25362 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:42.679468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:53073 (local address 127.15.208.62:44287)
0619 08:51:42.679535 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:42.679537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:42.679647 (+ 110us) server_negotiation.cc:407] Connection header received
0619 08:51:42.679688 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:42.679688 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:42.679701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:42.679752 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:42.680039 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.680789 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.681543 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.681724 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.681777 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:42.684175 (+ 2398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:53073: BlockingRecv error: recv got EOF from 127.15.208.2:53073 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 08:51:42 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:42 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:42.801635 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:42.797034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:54009)
0619 08:51:42.797199 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:42.797209 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:42.797285 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:42.797424 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:42.797424 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:42.797597 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:42.797643 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:42.797644 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.798272 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.798272 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:42.798819 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:42.798821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.799171 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.799172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:42.799222 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:42.799782 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:42.799787 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:42.801563 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
W20260619 08:51:42.801746 25362 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:42.797118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:54009 (local address 127.15.208.62:44287)
0619 08:51:42.797192 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:42.797194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:42.797288 (+ 94us) server_negotiation.cc:407] Connection header received
0619 08:51:42.797333 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:42.797333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:42.797347 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:42.797406 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:42.797715 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.798203 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.798896 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:42.799076 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:42.799131 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:42.801649 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:54009: BlockingRecv error: recv got EOF from 127.15.208.3:54009 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 08:51:43 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:43 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:43.571432 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:43.566830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:58019)
0619 08:51:43.566980 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:43.566991 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:43.567044 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:43.567240 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:43.567241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:43.567398 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:43.567448 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:43.567449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.568129 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.568129 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:43.568706 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:43.568708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.569129 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.569130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:43.569172 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:43.569634 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:43.569638 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:43.571366 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
W20260619 08:51:43.571511 25366 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:43.566978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:58019 (local address 127.15.208.62:44287)
0619 08:51:43.567110 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:43.567112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:43.567127 (+ 15us) server_negotiation.cc:407] Connection header received
0619 08:51:43.567149 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:43.567149 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:43.567159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:43.567221 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:43.567550 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.568058 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.568857 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.569064 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.569117 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:43.571458 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:58019: BlockingRecv error: recv got EOF from 127.15.208.1:58019 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
Jun 19 08:51:43 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:43 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:43.689399 25367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:43.684621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:43705)
0619 08:51:43.684797 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:43.684811 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:43.684903 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:43.685084 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:43.685085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:43.685266 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:43.685312 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:43.685314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.686153 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.686155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:43.686688 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:43.686690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.687066 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.687067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:43.687132 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:43.687555 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:43.687559 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:43.689331 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":92,"threads_started":1}
W20260619 08:51:43.689491 25366 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:43.684742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:43705 (local address 127.15.208.62:44287)
0619 08:51:43.684802 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:43.684804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:43.684889 (+ 85us) server_negotiation.cc:407] Connection header received
0619 08:51:43.684968 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:43.684969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:43.684981 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:43.685067 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:43.685410 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.686065 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.686781 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.686982 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.687068 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:43.689426 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:43705: BlockingRecv error: recv got EOF from 127.15.208.2:43705 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 08:51:43 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:43 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:43.806372 25368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:43.802140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:47047)
0619 08:51:43.802271 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:43.802291 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:43.802347 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:43.802472 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:43.802473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:43.802640 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:43.802688 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:43.802689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.803310 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.803311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:43.803822 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:43.803823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.804132 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.804133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:43.804170 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:43.804606 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:43.804610 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:43.806314 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
W20260619 08:51:43.806499 25366 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:43.802202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:47047 (local address 127.15.208.62:44287)
0619 08:51:43.802278 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:43.802280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:43.802351 (+ 71us) server_negotiation.cc:407] Connection header received
0619 08:51:43.802392 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:43.802392 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:43.802403 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:43.802456 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:43.802751 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.803245 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.803894 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:43.804070 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:43.804120 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:43.806400 (+ 2280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:47047: BlockingRecv error: recv got EOF from 127.15.208.3:47047 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 19 08:51:44 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:44 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:44.581238 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:44.572022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:43235)
0619 08:51:44.572176 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:44.572186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:44.572247 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:44.572381 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:44.572382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:44.572555 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:44.572605 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:44.572607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.573353 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.573353 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:44.573876 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:44.573877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.574211 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.574212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:44.574252 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:44.579113 (+ 4861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:44.579119 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:44.581178 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
W20260619 08:51:44.581390 25370 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:44.572114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:43235 (local address 127.15.208.62:44287)
0619 08:51:44.572238 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:44.572240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:44.572250 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:44.572294 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:44.572294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:44.572305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:44.572362 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:44.572679 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.573280 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.573948 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.574146 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.574197 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:44.581290 (+ 7093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:43235: BlockingRecv error: recv got EOF from 127.15.208.1:43235 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
Jun 19 08:51:44 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:44 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:44.694226 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:44.689899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:53601)
0619 08:51:44.690042 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:44.690070 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:44.690152 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:44.690291 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:44.690292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:44.690446 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:44.690493 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:44.690494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.691134 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.691135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:44.691630 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:44.691631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.691943 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.691944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:44.691983 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:44.692417 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:44.692421 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:44.694169 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
W20260619 08:51:44.694358 25370 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:44.689963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:53601 (local address 127.15.208.62:44287)
0619 08:51:44.690053 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:44.690055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:44.690152 (+ 97us) server_negotiation.cc:407] Connection header received
0619 08:51:44.690210 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:44.690211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:44.690222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:44.690273 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:44.690554 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.691074 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.691706 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.691885 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.691937 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:44.694273 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:53601: BlockingRecv error: recv got EOF from 127.15.208.2:53601 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 08:51:44 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:44 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:44.811480 25372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:44.806888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:40473)
0619 08:51:44.807071 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:44.807084 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:44.807176 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:44.807357 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:44.807358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:44.807598 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:44.807655 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:44.807658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.808292 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.808293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:44.808820 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:44.808822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.809121 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.809122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:44.809157 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:44.809612 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:44.809616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:44.811387 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
W20260619 08:51:44.811520 25370 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:44.806997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:40473 (local address 127.15.208.62:44287)
0619 08:51:44.807053 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:44.807055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:44.807158 (+ 103us) server_negotiation.cc:407] Connection header received
0619 08:51:44.807256 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:44.807257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:44.807274 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:44.807327 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:44.807737 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.808228 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.808885 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:44.809057 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:44.809109 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:44.811461 (+ 2352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:40473: BlockingRecv error: recv got EOF from 127.15.208.3:40473 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Jun 19 08:51:45 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:45 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:45.586385 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:45.581749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:60979)
0619 08:51:45.581892 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:45.581905 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:45.581982 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:45.582168 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:45.582169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:45.582345 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:45.582398 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:45.582400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.583053 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.583054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:45.583637 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:45.583639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.583985 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.583987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:45.584035 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:45.584529 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:45.584534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:45.586318 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":56,"threads_started":1}
W20260619 08:51:45.586472 25374 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:45.581833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:60979 (local address 127.15.208.62:44287)
0619 08:51:45.581981 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:45.581983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:45.581991 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:45.582033 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:45.582034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:45.582044 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:45.582102 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:45.582484 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.582980 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.583716 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.583900 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.583952 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:45.586418 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:60979: BlockingRecv error: recv got EOF from 127.15.208.1:60979 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
Jun 19 08:51:45 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:45 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:45.699843 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:45.694770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:36149)
0619 08:51:45.694926 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:45.694937 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:45.694992 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:45.695156 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:45.695157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:45.695332 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:45.695380 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:45.695381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.696081 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.696082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:45.696615 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:45.696617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.697157 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.697158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:45.697220 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:45.697803 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:45.697807 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:45.699725 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
W20260619 08:51:45.699884 25374 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:45.694844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:36149 (local address 127.15.208.62:44287)
0619 08:51:45.694966 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:45.694967 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:45.694981 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:51:45.695060 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:45.695060 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:45.695073 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:45.695134 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:45.695467 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.695997 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.696699 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.697061 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.697147 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:45.699816 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:36149: BlockingRecv error: recv got EOF from 127.15.208.2:36149 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 19 08:51:45 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:45 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:45.816398 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:45.812006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:59655)
0619 08:51:45.812182 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:45.812194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:45.812260 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:45.812403 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:45.812403 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:45.812559 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:45.812646 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:45.812649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.813346 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.813347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:45.813902 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:45.813904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.814240 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.814241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:45.814277 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:45.814694 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:45.814698 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:45.816341 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
W20260619 08:51:45.816570 25374 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:45.812168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:59655 (local address 127.15.208.62:44287)
0619 08:51:45.812219 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:45.812221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:45.812247 (+ 26us) server_negotiation.cc:407] Connection header received
0619 08:51:45.812334 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:45.812335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:45.812346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:45.812398 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:45.812766 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.813267 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.813984 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:45.814173 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:45.814228 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:45.816469 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:59655: BlockingRecv error: recv got EOF from 127.15.208.3:59655 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 08:51:46 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:46 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:46.591711 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:46.586946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:55351)
0619 08:51:46.587114 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:46.587125 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:46.587183 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:46.587338 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:46.587339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:46.587514 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:46.587565 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:46.587567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.588407 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.588408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:46.589074 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:46.589077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.589401 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.589401 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:46.589441 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:46.589895 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:46.589899 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:46.591643 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
W20260619 08:51:46.591813 25378 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:46.587092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:55351 (local address 127.15.208.62:44287)
0619 08:51:46.587220 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:46.587222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:46.587230 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:46.587252 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:46.587253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:46.587263 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:46.587322 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:46.587633 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.588321 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.589147 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.589335 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.589385 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:46.591738 (+ 2353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:55351: BlockingRecv error: recv got EOF from 127.15.208.1:55351 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
Jun 19 08:51:46 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:46 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:46.704792 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:46.700363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:45499)
0619 08:51:46.700517 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:46.700529 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:46.700598 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:46.700747 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:46.700748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:46.700915 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:46.700960 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:46.700961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.701592 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.701593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:46.702133 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:46.702134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.702448 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.702448 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:46.702487 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:46.702925 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:46.702930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:46.704669 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
W20260619 08:51:46.704890 25378 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:46.700446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:45499 (local address 127.15.208.62:44287)
0619 08:51:46.700497 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:46.700499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:46.700601 (+ 102us) server_negotiation.cc:407] Connection header received
0619 08:51:46.700657 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:46.700658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:46.700668 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:46.700710 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:46.701025 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.701518 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.702204 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.702383 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.702433 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:46.704824 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:45499: BlockingRecv error: recv got EOF from 127.15.208.2:45499 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 08:51:46 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:46 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:46.821252 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:46.816956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:41041)
0619 08:51:46.817086 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:46.817096 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:46.817152 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:46.817258 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:46.817259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:46.817420 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:46.817463 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:46.817465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.818079 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.818079 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:46.818605 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:46.818607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.818956 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.818957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:46.819001 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:46.819440 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:46.819444 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:46.821185 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
W20260619 08:51:46.821343 25378 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:46.817022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:41041 (local address 127.15.208.62:44287)
0619 08:51:46.817100 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:46.817102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:46.817150 (+ 48us) server_negotiation.cc:407] Connection header received
0619 08:51:46.817193 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:46.817194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:46.817205 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:46.817253 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:46.817521 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.818003 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.818693 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:46.818881 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:46.818945 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:46.821278 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:41041: BlockingRecv error: recv got EOF from 127.15.208.3:41041 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 08:51:47 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:47 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:47.596997 25381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:47.592316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:45507)
0619 08:51:47.592464 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:47.592482 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:47.592534 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:47.592768 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:47.592769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:47.592942 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:47.592987 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:47.592989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.593958 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.593959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:47.594486 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:47.594488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.594805 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.594806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:47.594845 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:47.595269 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:47.595273 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:47.596940 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
W20260619 08:51:47.597132 25382 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:47.592418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:45507 (local address 127.15.208.62:44287)
0619 08:51:47.592583 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:47.592588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:47.592602 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:51:47.592626 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:47.592626 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:47.592637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:47.592707 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:47.593105 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.593905 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.594560 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.594742 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.594793 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:47.597047 (+ 2254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:45507: BlockingRecv error: recv got EOF from 127.15.208.1:45507 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":118,"threads_started":1}
Jun 19 08:51:47 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:47 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:47.709875 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:47.705278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:46525)
0619 08:51:47.705460 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:47.705472 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:47.705542 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:47.705682 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:47.705683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:47.705847 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:47.705891 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:47.705892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.706564 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.706565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:47.707098 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:47.707099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.707421 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.707422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:47.707459 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:47.707919 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:47.707923 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:47.709818 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
W20260619 08:51:47.710017 25382 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:47.705404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:46525 (local address 127.15.208.62:44287)
0619 08:51:47.705490 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:47.705492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:47.705531 (+ 39us) server_negotiation.cc:407] Connection header received
0619 08:51:47.705602 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:47.705603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:47.705613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:47.705662 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:47.705989 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.706478 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.707178 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.707358 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.707410 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:47.709937 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:46525: BlockingRecv error: recv got EOF from 127.15.208.2:46525 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 08:51:47 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:47 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:47.826351 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:47.821720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:52583)
0619 08:51:47.821887 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:47.821898 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:47.821999 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:47.822163 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:47.822164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:47.822347 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:47.822389 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:47.822390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.823081 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.823082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:47.823644 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:47.823646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.824029 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.824030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:47.824098 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:47.824557 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:47.824561 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:47.826256 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
W20260619 08:51:47.826402 25382 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:47.821795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:52583 (local address 127.15.208.62:44287)
0619 08:51:47.821857 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:47.821859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:47.821962 (+ 103us) server_negotiation.cc:407] Connection header received
0619 08:51:47.822052 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:47.822052 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:47.822067 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:47.822133 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:47.822486 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.822995 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.823733 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:47.823924 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:47.824028 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:47.826336 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:52583: BlockingRecv error: recv got EOF from 127.15.208.3:52583 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 08:51:48 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:48 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:48.602458 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:48.597693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:34295)
0619 08:51:48.597863 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:48.597875 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:48.597925 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:48.598177 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:48.598178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:48.598415 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:48.598460 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:48.598462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.599166 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.599167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:48.599713 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:48.599715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.600103 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.600104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:48.600162 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:48.600621 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:48.600625 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:48.602386 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
W20260619 08:51:48.602627 25386 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:48.597804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:34295 (local address 127.15.208.62:44287)
0619 08:51:48.597914 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:48.597916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:48.597927 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:51:48.598048 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:48.598049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:48.598060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:48.598167 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:48.598579 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.599074 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.599805 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.600012 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.600104 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:48.602539 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:34295: BlockingRecv error: recv got EOF from 127.15.208.1:34295 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":51,"threads_started":1}
Jun 19 08:51:48 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:48 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:48.715072 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:48.710385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:43423)
0619 08:51:48.710525 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:48.710535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:48.710614 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:48.710745 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:48.710746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:48.710927 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:48.710970 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:48.710971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.711671 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.711672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:48.712263 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:48.712265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.712646 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.712647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:48.712702 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:48.713275 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:48.713279 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:48.714969 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
W20260619 08:51:48.715131 25386 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:48.710537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:43423 (local address 127.15.208.62:44287)
0619 08:51:48.710613 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:48.710616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:48.710625 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:51:48.710678 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:48.710679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:48.710690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:48.710743 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:48.711056 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.711577 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.712353 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.712572 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.712650 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:48.715070 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:43423: BlockingRecv error: recv got EOF from 127.15.208.2:43423 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 19 08:51:48 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:48 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:48.831533 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:48.826806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:57785)
0619 08:51:48.826961 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:48.826972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:48.827072 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:48.827220 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:48.827221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:48.827401 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:48.827444 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:48.827446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.828132 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.828133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:48.828707 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:48.828709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.829123 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.829124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:48.829185 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:48.829602 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:48.829607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:48.831448 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":53,"threads_started":1}
W20260619 08:51:48.831679 25386 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:48.826891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:57785 (local address 127.15.208.62:44287)
0619 08:51:48.826967 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:48.826970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:48.827038 (+ 68us) server_negotiation.cc:407] Connection header received
0619 08:51:48.827142 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:48.827143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:48.827158 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:48.827206 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:48.827524 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.828041 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.828824 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:48.829036 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:48.829122 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:48.831596 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:57785: BlockingRecv error: recv got EOF from 127.15.208.3:57785 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 08:51:49 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:49 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:49.611195 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:49.603028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:32987)
0619 08:51:49.603179 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:49.603191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:49.603276 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:49.603467 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:49.603468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:49.603657 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:49.603704 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:49.603705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.604336 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.604336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:49.604905 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:49.604908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.605246 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.605247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:49.605287 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:49.609227 (+ 3940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:49.609235 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:49.611134 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
W20260619 08:51:49.611335 25390 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:49.603080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:32987 (local address 127.15.208.62:44287)
0619 08:51:49.603231 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:49.603233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:49.603261 (+ 28us) server_negotiation.cc:407] Connection header received
0619 08:51:49.603347 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:49.603348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:49.603360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:49.603441 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:49.603777 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.604263 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.604993 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.605180 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.605233 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:49.611242 (+ 6009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:32987: BlockingRecv error: recv got EOF from 127.15.208.1:32987 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
Jun 19 08:51:49 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:49 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:49.720758 25391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:49.715615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:50497)
0619 08:51:49.715820 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:49.715832 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:49.715905 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:49.716064 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:49.716065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:49.716252 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:49.716296 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:49.716297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.717092 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.717093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:49.717622 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:49.717624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.718026 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.718026 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:49.718087 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:49.718578 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:49.718585 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:49.720647 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":111,"threads_started":1}
W20260619 08:51:49.720870 25390 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:49.715721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:50497 (local address 127.15.208.62:44287)
0619 08:51:49.715814 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:49.715816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:49.715886 (+ 70us) server_negotiation.cc:407] Connection header received
0619 08:51:49.715972 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:49.715973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:49.715985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:49.716058 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:49.716381 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.717007 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.717709 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.717918 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.718007 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:49.720787 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:50497: BlockingRecv error: recv got EOF from 127.15.208.2:50497 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 19 08:51:49 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:49 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.3@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:49.836870 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:49.832073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.3:52361)
0619 08:51:49.832216 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:49.832227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:49.832306 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:49.832438 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:49.832439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:49.832607 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:49.832656 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:49.832658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.833467 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.833468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:49.834020 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:49.834022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.834413 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.834414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:49.834471 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:49.835009 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:49.835013 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:49.836802 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
W20260619 08:51:49.836930 25390 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:49.832116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:52361 (local address 127.15.208.62:44287)
0619 08:51:49.832159 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:49.832161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:49.832319 (+ 158us) server_negotiation.cc:407] Connection header received
0619 08:51:49.832343 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:49.832343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:49.832355 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:49.832408 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:49.832879 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.833376 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.834101 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:49.834348 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:49.834401 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:49.836881 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.3:52361: BlockingRecv error: recv got EOF from 127.15.208.3:52361 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Jun 19 08:51:50 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:50 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.1@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:50.616611 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:50.611784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.1:50971)
0619 08:51:50.611941 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:50.611952 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:50.612006 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:50.612140 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:50.612140 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:50.612339 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:50.612388 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:50.612390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:50.613177 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:50.613178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:50.613776 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:50.613779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:50.614114 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:50.614115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:50.614155 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:50.614648 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:50.614653 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:50.616488 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
W20260619 08:51:50.616673 25394 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:50.611881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:50971 (local address 127.15.208.62:44287)
0619 08:51:50.611987 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:50.611989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:50.611997 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:50.612058 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:50.612059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:50.612069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:50.612127 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:50.612455 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:50.613109 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:50.613851 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:50.614039 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:50.614104 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:50.616572 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.1:50971: BlockingRecv error: recv got EOF from 127.15.208.1:50971 (error 108)
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":48,"threads_started":1}
Jun 19 08:51:50 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:51:50 dist-test-slave-p0h9 krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.208.2@KRBTEST.COM for oryx/127.15.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:51:50.725839 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:50.721556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44287 (local address 127.15.208.2:40225)
0619 08:51:50.721696 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:50.721712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:50.721765 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:50.721905 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:50.721906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:50.722069 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:50.722118 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:50.722119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:50.722738 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:50.722739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:50.723278 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:50.723280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:50.723608 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:50.723609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:50.723647 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:50.724058 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:50.724062 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:50.725771 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.208.62:44287: Server oryx/127.15.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
W20260619 08:51:50.725935 25394 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:50.721630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:40225 (local address 127.15.208.62:44287)
0619 08:51:50.721717 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:50.721718 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:50.721755 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:51:50.721823 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:50.721823 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:50.721834 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:50.721889 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:50.722182 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:50.722672 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:50.723357 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:50.723542 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:50.723599 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:50.725869 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.208.2:40225: BlockingRecv error: recv got EOF from 127.15.208.2:40225 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 08:51:50.760453 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 24919
I20260619 08:51:50.765179 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 25071
I20260619 08:51:50.769618 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 25203
I20260619 08:51:50.773823 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 24849
2026-06-19T08:51:50Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17402 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T08:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:50Z Disabled control of system clock
I20260619 08:51:50.801254 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39399
--webserver_interface=127.15.208.62
--webserver_port=0
--builtin_ntp_servers=127.15.208.20:35963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:39399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 08:51:50.883607 25399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:50.883822 25399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:50.883873 25399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:50.885104 25399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:50.885150 25399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:50.885165 25399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:50.885180 25399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:50.885195 25399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:50.886600 25399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:39399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:50.886797 25399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:50.887048 25399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:50.892185 25405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:50.892163 25404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:50.892279 25399 server_base.cc:1034] running on GCE node
W20260619 08:51:50.892170 25407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:50.892771 25399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:50.893131 25399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:50.894315 25399 hybrid_clock.cc:648] HybridClock initialized: now 1781859110894303 us; error 57 us; skew 500 ppm
I20260619 08:51:50.895542 25399 webserver.cc:460] Webserver started at http://127.15.208.62:38599/ using document root <none> and password file <none>
I20260619 08:51:50.895779 25399 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:50.895824 25399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:50.895973 25399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:50.897153 25399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "9a830cbf1ab5479c9b1809de26a33498"
format_stamp: "Formatted at 2026-06-19 08:51:50 on dist-test-slave-p0h9"
server_key: "c52e6ba52f4a9e449da2fc42aa8ff02b"
server_key_iv: "00eedc13881001721a40fd795d5258ac"
server_key_version: "encryptionkey@0"
I20260619 08:51:50.897513 25399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "9a830cbf1ab5479c9b1809de26a33498"
format_stamp: "Formatted at 2026-06-19 08:51:50 on dist-test-slave-p0h9"
server_key: "c52e6ba52f4a9e449da2fc42aa8ff02b"
server_key_iv: "00eedc13881001721a40fd795d5258ac"
server_key_version: "encryptionkey@0"
I20260619 08:51:50.898784 25399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:51:50.899462 25413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:50.899649 25399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:50.899744 25399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "9a830cbf1ab5479c9b1809de26a33498"
format_stamp: "Formatted at 2026-06-19 08:51:50 on dist-test-slave-p0h9"
server_key: "c52e6ba52f4a9e449da2fc42aa8ff02b"
server_key_iv: "00eedc13881001721a40fd795d5258ac"
server_key_version: "encryptionkey@0"
I20260619 08:51:50.899825 25399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:50.921890 25399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:50.922273 25399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:50.922426 25399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:50.926446 25399 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:39399
I20260619 08:51:50.926484 25465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:39399 every 8 connection(s)
I20260619 08:51:50.926860 25399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:50.927582 25466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:50.930239 25466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498: Bootstrap starting.
I20260619 08:51:50.930908 25466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:50.931219 25466 log.cc:824] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:50.931902 25466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498: No bootstrap required, opened a new log
I20260619 08:51:50.933560 25466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a830cbf1ab5479c9b1809de26a33498" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39399 } }
I20260619 08:51:50.933696 25466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:50.933741 25466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a830cbf1ab5479c9b1809de26a33498, State: Initialized, Role: FOLLOWER
I20260619 08:51:50.933892 25466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a830cbf1ab5479c9b1809de26a33498" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39399 } }
I20260619 08:51:50.933959 25466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:50.934007 25466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:50.934056 25466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:50.934577 25466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a830cbf1ab5479c9b1809de26a33498" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39399 } }
I20260619 08:51:50.934690 25466 leader_election.cc:302] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a830cbf1ab5479c9b1809de26a33498; no voters:
I20260619 08:51:50.934880 25466 leader_election.cc:288] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:50.934948 25469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:50.935123 25469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [term 1 LEADER]: Becoming Leader. State: Replica: 9a830cbf1ab5479c9b1809de26a33498, State: Running, Role: LEADER
I20260619 08:51:50.935155 25466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:50.935225 25469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a830cbf1ab5479c9b1809de26a33498" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39399 } }
I20260619 08:51:50.935591 25471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a830cbf1ab5479c9b1809de26a33498. Latest consensus state: current_term: 1 leader_uuid: "9a830cbf1ab5479c9b1809de26a33498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a830cbf1ab5479c9b1809de26a33498" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39399 } } }
I20260619 08:51:50.935688 25471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:50.935721 25470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a830cbf1ab5479c9b1809de26a33498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a830cbf1ab5479c9b1809de26a33498" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39399 } } }
I20260619 08:51:50.935778 25470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:50.936000 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 25399
I20260619 08:51:50.936034 25477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:50.936112 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:50.936353 16192 external_mini_cluster.cc:1428] Setting key ef04418f0560b46eb788d66880a5da01
I20260619 08:51:50.936594 25477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:50.939106 25477 catalog_manager.cc:1347] Generated new cluster ID: 565ff5fd1d524fdebe466249ca888afe
I20260619 08:51:50.939163 25477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:50.942231 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:50.936928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.15.208.62:39399)
0619 08:51:50.937223 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:50.937231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:50.937430 (+ 199us) server_negotiation.cc:407] Connection header received
0619 08:51:50.937575 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:50.937582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:50.937651 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:50.937796 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:50.937985 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:50.938939 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:50.940014 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:50.940673 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:50.940831 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:50.941513 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:50.941516 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:50.941517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:50.941724 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:50.941764 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:50.941777 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:50.941849 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:50.941915 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 08:51:50.941997 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":70,"threads_started":1}
I20260619 08:51:50.952591 25477 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:50.953106 25477 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:50.961625 25477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a830cbf1ab5479c9b1809de26a33498: Generated new TSK 0
I20260619 08:51:50.961812 25477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:51:50.968010 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39399
--builtin_ntp_servers=127.15.208.20:35963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 08:51:51.049831 25490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:51.050068 25490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:51.050105 25490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:51.051281 25490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:51.051347 25490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:51.051391 25490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:51.052775 25490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:51.053053 25490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:51.053360 25490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:51.059130 25496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:51.059139 25498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:51.059144 25495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.059821 25490 server_base.cc:1034] running on GCE node
I20260619 08:51:51.060032 25490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:51.060277 25490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:51.061440 25490 hybrid_clock.cc:648] HybridClock initialized: now 1781859111061421 us; error 33 us; skew 500 ppm
I20260619 08:51:51.062947 25490 webserver.cc:460] Webserver started at http://127.15.208.1:41799/ using document root <none> and password file <none>
I20260619 08:51:51.063153 25490 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:51.063208 25490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:51.063298 25490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:51.064306 25490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "c84a86d4b68e4d23a37a37c49b0d120b"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "8a065992532002877fa97ee22112c682"
server_key_iv: "b348a35d38a36e80e0b3296b11ed6fe1"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.064687 25490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "c84a86d4b68e4d23a37a37c49b0d120b"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "8a065992532002877fa97ee22112c682"
server_key_iv: "b348a35d38a36e80e0b3296b11ed6fe1"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.065985 25490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:51.066640 25504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.066807 25490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:51.066891 25490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "c84a86d4b68e4d23a37a37c49b0d120b"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "8a065992532002877fa97ee22112c682"
server_key_iv: "b348a35d38a36e80e0b3296b11ed6fe1"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.066944 25490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:51.083590 25490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:51.083853 25490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:51.083956 25490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:51.084139 25490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:51.084470 25490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:51.084503 25490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.084565 25490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:51.084599 25490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.090557 25490 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:44159
I20260619 08:51:51.090615 25617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:44159 every 8 connection(s)
I20260619 08:51:51.090948 25490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:51.093180 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 25490
I20260619 08:51:51.093289 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:51.093530 16192 external_mini_cluster.cc:1428] Setting key a02c73b8790a28ad558354c80b38eca8
I20260619 08:51:51.094621 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39399
--builtin_ntp_servers=127.15.208.20:35963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 08:51:51.097465 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.091851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:53019 (local address 127.15.208.62:39399)
0619 08:51:51.091922 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:51.091924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:51.092232 (+ 308us) server_negotiation.cc:407] Connection header received
0619 08:51:51.092967 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:51.092969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:51.092998 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:51.093108 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:51.093953 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.094589 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.095657 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.095976 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.096055 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:51.097161 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:51.097163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:51.097163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:51.097168 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:51.097188 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:51.097194 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:51.097256 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:51.097355 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 08:51:51.097385 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:51:51.097693 25620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.091839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39399 (local address 127.15.208.1:53019)
0619 08:51:51.092080 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:51.092130 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:51.092832 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:51.093194 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:51.093198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:51.093206 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:51.093821 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:51.093829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.094708 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.094709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:51.095562 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:51.095565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.096131 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.096132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:51.096298 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:51.096976 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:51.096977 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:51.097023 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:51.097061 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:51.097268 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:51.097271 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:51.097274 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:51.097354 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:51:51.097457 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":102,"threads_started":1}
I20260619 08:51:51.098217 25618 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39399
I20260619 08:51:51.098333 25618 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:51.098531 25618 heartbeater.cc:511] Master 127.15.208.62:39399 requested a full tablet report, sending...
I20260619 08:51:51.099548 25430 ts_manager.cc:194] Registered new tserver with Master: c84a86d4b68e4d23a37a37c49b0d120b (127.15.208.1:44159)
I20260619 08:51:51.100168 25430 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.1:53019
W20260619 08:51:51.179423 25621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:51.179649 25621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:51.179684 25621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:51.180904 25621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:51.180966 25621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:51.181007 25621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:51.182327 25621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:51.182578 25621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:51.182865 25621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:51.187765 25629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:51.187849 25626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:51.187775 25627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.188043 25621 server_base.cc:1034] running on GCE node
I20260619 08:51:51.188212 25621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:51.188386 25621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:51.189522 25621 hybrid_clock.cc:648] HybridClock initialized: now 1781859111189504 us; error 31 us; skew 500 ppm
I20260619 08:51:51.190778 25621 webserver.cc:460] Webserver started at http://127.15.208.2:45737/ using document root <none> and password file <none>
I20260619 08:51:51.191000 25621 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:51.191043 25621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:51.191208 25621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:51.192266 25621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "5e19d9e6d54b4f95955db1d04e9d8da0"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "82250b54515dbf812a6ec8fa770bc456"
server_key_iv: "7199c88cdec26092b31dc98b2d565f4d"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.192657 25621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "5e19d9e6d54b4f95955db1d04e9d8da0"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "82250b54515dbf812a6ec8fa770bc456"
server_key_iv: "7199c88cdec26092b31dc98b2d565f4d"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.194005 25621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:51.194727 25635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.194895 25621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:51.194972 25621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "5e19d9e6d54b4f95955db1d04e9d8da0"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "82250b54515dbf812a6ec8fa770bc456"
server_key_iv: "7199c88cdec26092b31dc98b2d565f4d"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.195024 25621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:51.214895 25621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:51.215174 25621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:51.215276 25621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:51.215469 25621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:51.215840 25621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:51.215876 25621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.215938 25621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:51.215966 25621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.221657 25621 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:42177
I20260619 08:51:51.221698 25748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:42177 every 8 connection(s)
I20260619 08:51:51.222029 25621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:51:51.227020 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.222747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:40797 (local address 127.15.208.62:39399)
0619 08:51:51.222812 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:51.222814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:51.223089 (+ 275us) server_negotiation.cc:407] Connection header received
0619 08:51:51.223586 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:51.223587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:51.223596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:51.223660 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:51.224306 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.224802 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.225741 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.225934 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.226012 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:51.226785 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:51.226785 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:51.226786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:51.226789 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:51.226800 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:51.226803 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:51.226846 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:51.226923 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 08:51:51.226962 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:51:51.227259 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.222744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39399 (local address 127.15.208.2:40797)
0619 08:51:51.222932 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:51.222970 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:51.223497 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:51.223722 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:51.223725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:51.223734 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:51.224227 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:51.224231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.224878 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.224878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:51.225616 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:51.225618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.226028 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.226029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:51.226156 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:51.226659 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:51.226659 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:51.226687 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:51.226708 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:51.226858 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:51.226861 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:51.226863 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:51.226912 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:51:51.226987 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260619 08:51:51.227792 25749 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39399
I20260619 08:51:51.227870 25749 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:51.228075 25749 heartbeater.cc:511] Master 127.15.208.62:39399 requested a full tablet report, sending...
I20260619 08:51:51.228554 25430 ts_manager.cc:194] Registered new tserver with Master: 5e19d9e6d54b4f95955db1d04e9d8da0 (127.15.208.2:42177)
I20260619 08:51:51.228968 25430 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.2:40797
I20260619 08:51:51.230654 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 25621
I20260619 08:51:51.230752 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:51.230906 16192 external_mini_cluster.cc:1428] Setting key a80f217e7b7795ab0044e2d05d21ee7c
I20260619 08:51:51.231753 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39399
--builtin_ntp_servers=127.15.208.20:35963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 08:51:51.312144 25752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:51.312352 25752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:51.312373 25752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:51.313477 25752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:51.313512 25752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:51.313536 25752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:51.314762 25752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:51.314945 25752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:51.315186 25752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:51.319965 25760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:51.319979 25757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:51.319983 25758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.320267 25752 server_base.cc:1034] running on GCE node
I20260619 08:51:51.320449 25752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:51.320675 25752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:51.321820 25752 hybrid_clock.cc:648] HybridClock initialized: now 1781859111321807 us; error 28 us; skew 500 ppm
I20260619 08:51:51.323041 25752 webserver.cc:460] Webserver started at http://127.15.208.3:45199/ using document root <none> and password file <none>
I20260619 08:51:51.323302 25752 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:51.323374 25752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:51.323545 25752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:51.324575 25752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "36b3e0a67fef47f895d636d781dee499"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "bf17987c435ceaaa27e758f4b51d39e8"
server_key_iv: "6ea10fcdd5064a1c0a8fd3083b0daab2"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.324955 25752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "36b3e0a67fef47f895d636d781dee499"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "bf17987c435ceaaa27e758f4b51d39e8"
server_key_iv: "6ea10fcdd5064a1c0a8fd3083b0daab2"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.326272 25752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:51.327025 25766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.327220 25752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:51.327288 25752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "36b3e0a67fef47f895d636d781dee499"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "bf17987c435ceaaa27e758f4b51d39e8"
server_key_iv: "6ea10fcdd5064a1c0a8fd3083b0daab2"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.327370 25752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:51.342245 25752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:51.342590 25752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:51.342747 25752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:51.342980 25752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:51.343353 25752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:51.343387 25752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.343432 25752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:51.343472 25752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.349037 25752 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:33701
I20260619 08:51:51.349128 25879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:33701 every 8 connection(s)
I20260619 08:51:51.349462 25752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:51.354624 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.350214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:57091 (local address 127.15.208.62:39399)
0619 08:51:51.350284 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:51.350286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:51.350514 (+ 228us) server_negotiation.cc:407] Connection header received
0619 08:51:51.351156 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:51.351157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:51.351167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:51.351219 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:51.351835 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.352349 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.353394 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.353589 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.353642 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:51.354374 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:51.354375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:51.354375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:51.354380 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:51.354393 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:51.354397 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:51.354438 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:51.354536 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 08:51:51.354565 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:51:51.354853 25882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.350213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39399 (local address 127.15.208.3:57091)
0619 08:51:51.350405 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:51.350438 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:51.351062 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:51.351272 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:51.351275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:51.351284 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:51.351766 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:51.351770 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.352431 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.352431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:51.353319 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:51.353322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:51.353663 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:51.353663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:51.353767 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:51.354253 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:51.354253 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:51.354279 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:51.354300 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:51.354449 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:51.354453 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:51.354456 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:51.354528 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 08:51:51.354606 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260619 08:51:51.355347 25880 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39399
I20260619 08:51:51.355437 25880 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:51.355594 25880 heartbeater.cc:511] Master 127.15.208.62:39399 requested a full tablet report, sending...
I20260619 08:51:51.355953 25430 ts_manager.cc:194] Registered new tserver with Master: 36b3e0a67fef47f895d636d781dee499 (127.15.208.3:33701)
I20260619 08:51:51.356280 25430 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.3:57091
I20260619 08:51:51.356603 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 25752
I20260619 08:51:51.356699 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:51.356950 16192 external_mini_cluster.cc:1428] Setting key 953db2566976c0800dcd72de9f3713c2
I20260619 08:51:51.357893 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 08:51:51.359638 25486 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:51.359208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.15.208.62:39399)
0619 08:51:51.359293 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:51.359295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:51.359336 (+ 41us) server_negotiation.cc:407] Connection header received
0619 08:51:51.359400 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:51.359400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:51.359409 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:51.359449 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:51.359573 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53668: BlockingRecv error: recv got EOF from 127.0.0.1:53668 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 08:51:51.359740 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 25490
I20260619 08:51:51.364861 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 25621
I20260619 08:51:51.369870 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 25752
I20260619 08:51:51.374346 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 25399
2026-06-19T08:51:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (599 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T08:51:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:51Z Disabled control of system clock
I20260619 08:51:51.400298 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39743
--webserver_interface=127.15.208.62
--webserver_port=0
--builtin_ntp_servers=127.15.208.20:41999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:39743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 08:51:51.479880 25891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:51.480074 25891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:51.480095 25891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:51.481161 25891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:51.481196 25891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:51.481209 25891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:51.481221 25891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:51.481233 25891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:51.482473 25891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:39743
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:51.482666 25891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:51.482896 25891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:51.487869 25897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:51.487994 25899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.487849 25891 server_base.cc:1034] running on GCE node
W20260619 08:51:51.487849 25896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.488325 25891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:51.488633 25891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:51.489801 25891 hybrid_clock.cc:648] HybridClock initialized: now 1781859111489811 us; error 63 us; skew 500 ppm
I20260619 08:51:51.490988 25891 webserver.cc:460] Webserver started at http://127.15.208.62:39943/ using document root <none> and password file <none>
I20260619 08:51:51.491222 25891 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:51.491289 25891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:51.491439 25891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:51.492478 25891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "c3a7299ccbfc42508500035ce7e3db39"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "a563b8048d50b7ab2018ef5f9774a124"
server_key_iv: "f672d880214f767dcc7752965dd2467c"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.492893 25891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "c3a7299ccbfc42508500035ce7e3db39"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "a563b8048d50b7ab2018ef5f9774a124"
server_key_iv: "f672d880214f767dcc7752965dd2467c"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.494292 25891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:51.495072 25905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.495266 25891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:51.495373 25891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "c3a7299ccbfc42508500035ce7e3db39"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "a563b8048d50b7ab2018ef5f9774a124"
server_key_iv: "f672d880214f767dcc7752965dd2467c"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.495448 25891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:51.498891 25891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:51.499158 25891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:51.499313 25891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:51.503187 25891 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:39743
I20260619 08:51:51.503235 25957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:39743 every 8 connection(s)
I20260619 08:51:51.503573 25891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:51.504211 25958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:51.504218 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 25891
I20260619 08:51:51.504329 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:51.504527 16192 external_mini_cluster.cc:1428] Setting key 8f49922ea77a9d810a32c575bd5e8b0e
I20260619 08:51:51.507548 25961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.505101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60296 (local address 127.15.208.62:39743)
0619 08:51:51.505322 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:51.505332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:51.505367 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:51:51.506248 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:51.506258 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:51.506312 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:51.506489 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:51.507112 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:51.507116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:51.507117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:51.507121 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:51.507166 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:51.507179 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:51.507252 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:51.507310 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 08:51:51.507364 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260619 08:51:51.508473 25958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39: Bootstrap starting.
I20260619 08:51:51.509127 25958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:51.509413 25958 log.cc:824] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:51.510099 25958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39: No bootstrap required, opened a new log
I20260619 08:51:51.511409 25958 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a7299ccbfc42508500035ce7e3db39" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39743 } }
I20260619 08:51:51.511567 25958 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:51.511619 25958 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3a7299ccbfc42508500035ce7e3db39, State: Initialized, Role: FOLLOWER
I20260619 08:51:51.511758 25958 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a7299ccbfc42508500035ce7e3db39" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39743 } }
I20260619 08:51:51.511864 25958 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:51.511910 25958 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:51.511992 25958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:51.512555 25958 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a7299ccbfc42508500035ce7e3db39" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39743 } }
I20260619 08:51:51.512674 25958 leader_election.cc:302] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3a7299ccbfc42508500035ce7e3db39; no voters:
I20260619 08:51:51.512903 25958 leader_election.cc:288] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:51.513195 25963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:51.513222 25958 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:51.513336 25963 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [term 1 LEADER]: Becoming Leader. State: Replica: c3a7299ccbfc42508500035ce7e3db39, State: Running, Role: LEADER
I20260619 08:51:51.513437 25963 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a7299ccbfc42508500035ce7e3db39" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39743 } }
I20260619 08:51:51.513811 25965 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3a7299ccbfc42508500035ce7e3db39. Latest consensus state: current_term: 1 leader_uuid: "c3a7299ccbfc42508500035ce7e3db39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a7299ccbfc42508500035ce7e3db39" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39743 } } }
I20260619 08:51:51.513806 25964 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3a7299ccbfc42508500035ce7e3db39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a7299ccbfc42508500035ce7e3db39" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39743 } } }
I20260619 08:51:51.513921 25965 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:51.513949 25964 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:51.514183 25972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:51.514794 25972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:51.516413 25972 catalog_manager.cc:1347] Generated new cluster ID: 57878d50f9ed4a5ebabfb18e33fc9581
I20260619 08:51:51.516469 25972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:51.528296 25972 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:51.528476 25972 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:51.531607 25972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3a7299ccbfc42508500035ce7e3db39: Generated new TSK 0
I20260619 08:51:51.531774 25972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:51:51.534598 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39743
--builtin_ntp_servers=127.15.208.20:41999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 08:51:51.616302 25982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:51.616500 25982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:51.616523 25982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:51.617740 25982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:51.617784 25982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:51.617811 25982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:51.619222 25982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:51.619477 25982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:51.619789 25982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:51.624603 25987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:51.624693 25988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.624848 25982 server_base.cc:1034] running on GCE node
W20260619 08:51:51.624887 25990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.625114 25982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:51.625380 25982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:51.626554 25982 hybrid_clock.cc:648] HybridClock initialized: now 1781859111626524 us; error 45 us; skew 500 ppm
I20260619 08:51:51.627971 25982 webserver.cc:460] Webserver started at http://127.15.208.1:32899/ using document root <none> and password file <none>
I20260619 08:51:51.628302 25982 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:51.628364 25982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:51.628507 25982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:51.630030 25982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "f299a9bcf209484c8c25401d72c23905"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "2c8a58d99695d0355152cc28073dce33"
server_key_iv: "cf34cb22d80546721924c58c9a9aa278"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.630475 25982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "f299a9bcf209484c8c25401d72c23905"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "2c8a58d99695d0355152cc28073dce33"
server_key_iv: "cf34cb22d80546721924c58c9a9aa278"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.632149 25982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:51.633008 25996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.633164 25982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:51.633235 25982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "f299a9bcf209484c8c25401d72c23905"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "2c8a58d99695d0355152cc28073dce33"
server_key_iv: "cf34cb22d80546721924c58c9a9aa278"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.633297 25982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:51.637081 25982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:51.637342 25982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:51.637486 25982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:51.637727 25982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:51.638041 25982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:51.638074 25982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.638094 25982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:51.638141 25982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.643726 25982 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:34123
I20260619 08:51:51.643807 26109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:34123 every 8 connection(s)
I20260619 08:51:51.644137 25982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:51.646988 25961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.644913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:37259 (local address 127.15.208.62:39743)
0619 08:51:51.644979 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:51.644982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:51.645221 (+ 239us) server_negotiation.cc:407] Connection header received
0619 08:51:51.645717 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:51.645718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:51.645723 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:51.645779 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:51.646811 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:51.646813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:51.646813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:51.646816 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:51.646830 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:51.646833 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:51.646878 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:51.646922 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:51:51.646934 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:51:51.647202 26112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.644922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39743 (local address 127.15.208.1:37259)
0619 08:51:51.645116 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:51.645149 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:51.645626 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:51.645832 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:51.645835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:51.645843 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:51.646677 (+ 834us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:51.646677 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:51.646704 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:51.646729 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:51.646876 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:51.646878 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:51.646880 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:51.646929 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:51:51.646975 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":105,"threads_started":1}
I20260619 08:51:51.647701 26110 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39743
I20260619 08:51:51.647809 26110 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:51.647961 26110 heartbeater.cc:511] Master 127.15.208.62:39743 requested a full tablet report, sending...
I20260619 08:51:51.648437 25922 ts_manager.cc:194] Registered new tserver with Master: f299a9bcf209484c8c25401d72c23905 (127.15.208.1:34123)
I20260619 08:51:51.649405 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 25982
I20260619 08:51:51.649516 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:51.649735 16192 external_mini_cluster.cc:1428] Setting key 06a072f3bcbffa1f7b78e6022d17e419
I20260619 08:51:51.650593 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39743
--builtin_ntp_servers=127.15.208.20:41999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 08:51:51.729259 26113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:51.729478 26113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:51.729522 26113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:51.730687 26113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:51.730753 26113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:51.730794 26113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:51.732101 26113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:51.732354 26113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:51.732648 26113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:51.737433 26118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.737577 26113 server_base.cc:1034] running on GCE node
W20260619 08:51:51.737457 26119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:51.737460 26121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.737872 26113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:51.738121 26113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:51.739305 26113 hybrid_clock.cc:648] HybridClock initialized: now 1781859111739277 us; error 47 us; skew 500 ppm
I20260619 08:51:51.740679 26113 webserver.cc:460] Webserver started at http://127.15.208.2:45331/ using document root <none> and password file <none>
I20260619 08:51:51.740949 26113 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:51.741001 26113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:51.741093 26113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:51.742093 26113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "74b12c2c1a0d4a3e84601433b7c08515"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "fb3ab23cb73d741f0b91cfe51fbbc98d"
server_key_iv: "17fa8db94d4458a22e156d58269c462b"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.742389 26113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "74b12c2c1a0d4a3e84601433b7c08515"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "fb3ab23cb73d741f0b91cfe51fbbc98d"
server_key_iv: "17fa8db94d4458a22e156d58269c462b"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.743634 26113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:51.744293 26127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.744474 26113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:51.744570 26113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "74b12c2c1a0d4a3e84601433b7c08515"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "fb3ab23cb73d741f0b91cfe51fbbc98d"
server_key_iv: "17fa8db94d4458a22e156d58269c462b"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.744657 26113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:51.747943 26113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:51.748193 26113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:51.748340 26113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:51.748551 26113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:51.748903 26113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:51.748955 26113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.748993 26113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:51.749020 26113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.754861 26113 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:34269
I20260619 08:51:51.754949 26240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:34269 every 8 connection(s)
I20260619 08:51:51.755264 26113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:51:51.758244 25961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.756040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:48209 (local address 127.15.208.62:39743)
0619 08:51:51.756136 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:51.756138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:51.756325 (+ 187us) server_negotiation.cc:407] Connection header received
0619 08:51:51.756808 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:51.756809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:51.756821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:51.756902 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:51.758036 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:51.758037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:51.758037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:51.758041 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:51.758055 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:51.758058 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:51.758103 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:51.758178 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 08:51:51.758189 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 08:51:51.758463 26243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.756013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39743 (local address 127.15.208.2:48209)
0619 08:51:51.756193 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:51.756231 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:51.756668 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:51.756960 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:51.756964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:51.756974 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:51.757888 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:51.757888 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:51.757917 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:51.757946 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:51.758103 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:51.758105 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:51.758107 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:51:51.758176 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 08:51:51.758220 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260619 08:51:51.759028 26241 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39743
I20260619 08:51:51.759132 26241 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:51.759299 26241 heartbeater.cc:511] Master 127.15.208.62:39743 requested a full tablet report, sending...
I20260619 08:51:51.759750 25922 ts_manager.cc:194] Registered new tserver with Master: 74b12c2c1a0d4a3e84601433b7c08515 (127.15.208.2:34269)
I20260619 08:51:51.765209 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 26113
I20260619 08:51:51.765290 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:51.765455 16192 external_mini_cluster.cc:1428] Setting key d11098169d175e3521bbe5cf3591e3a7
I20260619 08:51:51.766368 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39743
--builtin_ntp_servers=127.15.208.20:41999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 08:51:51.851145 26244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:51.851402 26244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:51.851436 26244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:51.852667 26244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:51.852730 26244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:51.852792 26244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:51.854117 26244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:51.854368 26244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:51.854655 26244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:51.859534 26250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:51.859570 26249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.859591 26244 server_base.cc:1034] running on GCE node
W20260619 08:51:51.859570 26252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:51.859994 26244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:51.860234 26244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:51.861392 26244 hybrid_clock.cc:648] HybridClock initialized: now 1781859111861373 us; error 37 us; skew 500 ppm
I20260619 08:51:51.862665 26244 webserver.cc:460] Webserver started at http://127.15.208.3:43791/ using document root <none> and password file <none>
I20260619 08:51:51.862895 26244 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:51.862958 26244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:51.863095 26244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:51.864209 26244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "4be63c4dea364ec398fa478c41b9f318"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "d4e56d97a4adb0f953cf9ac34751ecd9"
server_key_iv: "8aeb8a03e85a7201352f39482f70a539"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.864567 26244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "4be63c4dea364ec398fa478c41b9f318"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "d4e56d97a4adb0f953cf9ac34751ecd9"
server_key_iv: "8aeb8a03e85a7201352f39482f70a539"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.866604 26244 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260619 08:51:51.867379 26258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.867590 26244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:51.867687 26244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "4be63c4dea364ec398fa478c41b9f318"
format_stamp: "Formatted at 2026-06-19 08:51:51 on dist-test-slave-p0h9"
server_key: "d4e56d97a4adb0f953cf9ac34751ecd9"
server_key_iv: "8aeb8a03e85a7201352f39482f70a539"
server_key_version: "encryptionkey@0"
I20260619 08:51:51.867767 26244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:51.871156 26244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:51.871410 26244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:51.871551 26244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:51.871744 26244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:51.872032 26244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:51.872081 26244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.872117 26244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:51.872145 26244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:51.877835 26244 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:34785
I20260619 08:51:51.877885 26371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:34785 every 8 connection(s)
I20260619 08:51:51.878283 26244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:51:51.881068 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 26244
I20260619 08:51:51.881176 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:51.881407 16192 external_mini_cluster.cc:1428] Setting key fecf47bd8e879ad379e5b0e96d7bc6f3
I20260619 08:51:51.881526 25961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.879050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:59185 (local address 127.15.208.62:39743)
0619 08:51:51.879114 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:51.879116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:51.879333 (+ 217us) server_negotiation.cc:407] Connection header received
0619 08:51:51.879905 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:51.879906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:51.879913 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:51.879969 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:51.881201 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:51.881203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:51.881203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:51:51.881208 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:51.881224 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:51:51.881229 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:51.881282 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:51.881401 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 08:51:51.881414 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:51:51.881737 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:51.879036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39743 (local address 127.15.208.3:59185)
0619 08:51:51.879218 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:51.879250 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:51.879772 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:51.880048 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:51.880052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:51.880061 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:51.881047 (+ 986us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:51:51.881048 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:51.881077 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:51:51.881105 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:51.881302 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:51.881309 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:51.881313 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:51:51.881412 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 08:51:51.881511 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260619 08:51:51.882148 26372 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39743
I20260619 08:51:51.882244 26372 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:51.882388 26372 heartbeater.cc:511] Master 127.15.208.62:39743 requested a full tablet report, sending...
I20260619 08:51:51.882723 25921 ts_manager.cc:194] Registered new tserver with Master: 4be63c4dea364ec398fa478c41b9f318 (127.15.208.3:34785)
I20260619 08:51:51.893168 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 08:51:51.895072 25961 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:51:51.894490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60304 (local address 127.15.208.62:39743)
0619 08:51:51.894549 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:51.894552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:51.894741 (+ 189us) server_negotiation.cc:407] Connection header received
0619 08:51:51.894785 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:51.894786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:51.894790 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:51.894837 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:51.894996 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60304: BlockingRecv error: recv got EOF from 127.0.0.1:60304 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:51:51.895100 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 25982
I20260619 08:51:51.899772 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 26113
I20260619 08:51:51.903901 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 26244
I20260619 08:51:51.907814 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 25891
2026-06-19T08:51:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (533 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:51 dist-test-slave-p0h9 krb5kdc[26382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:51 dist-test-slave-p0h9 krb5kdc[26382](info): set up 2 sockets
Jun 19 08:51:51 dist-test-slave-p0h9 krb5kdc[26382](info): commencing operation
krb5kdc: starting...
W20260619 08:51:52.950095 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:51:53.065423 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:44801
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:44801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:53.144377 26395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:53.144624 26395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:53.144662 26395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:53.145846 26395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:51:53.145915 26395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:53.145946 26395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:53.145974 26395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:51:53.146024 26395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:51:53.147478 26395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:44801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:44801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:53.147764 26395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:53.147989 26395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:53.152751 26403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:53.152753 26400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:53.152767 26401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:53.152971 26395 server_base.cc:1034] running on GCE node
I20260619 08:51:53.153328 26395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:53.153577 26395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:53.154735 26395 hybrid_clock.cc:648] HybridClock initialized: now 1781859113154719 us; error 27 us; skew 500 ppm
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859113, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:53.157196 26395 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:51:53.157673 26395 webserver.cc:460] Webserver started at http://127.15.208.62:33687/ using document root <none> and password file <none>
I20260619 08:51:53.157905 26395 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:53.157977 26395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:53.158139 26395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:53.159247 26395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "40b6c4de61634fe9b4e752c964d12767"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "70efcacad7b1533fae71b9784bef2cf9"
server_key_iv: "c4c3b92c7e7691dfb336dd2467845911"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.159608 26395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "40b6c4de61634fe9b4e752c964d12767"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "70efcacad7b1533fae71b9784bef2cf9"
server_key_iv: "c4c3b92c7e7691dfb336dd2467845911"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.160957 26395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:51:53.161738 26410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:53.161926 26395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:51:53.162040 26395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "40b6c4de61634fe9b4e752c964d12767"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "70efcacad7b1533fae71b9784bef2cf9"
server_key_iv: "c4c3b92c7e7691dfb336dd2467845911"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.162114 26395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:53.182430 26395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:53.184298 26395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:53.184455 26395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:53.188524 26395 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:44801
I20260619 08:51:53.188577 26462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:44801 every 8 connection(s)
I20260619 08:51:53.188943 26395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:51:53.189850 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 26395
I20260619 08:51:53.189904 26463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:53.189968 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:51:53.190160 16192 external_mini_cluster.cc:1428] Setting key 5ac5e0e0fd9b7915845b935261c506d3
I20260619 08:51:53.193194 26463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767: Bootstrap starting.
I20260619 08:51:53.193888 26463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:53.194183 26463 log.cc:824] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:53.194857 26463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767: No bootstrap required, opened a new log
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:53.196856 26463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b6c4de61634fe9b4e752c964d12767" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44801 } }
I20260619 08:51:53.197028 26463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:53.197088 26463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b6c4de61634fe9b4e752c964d12767, State: Initialized, Role: FOLLOWER
I20260619 08:51:53.197224 26463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b6c4de61634fe9b4e752c964d12767" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44801 } }
I20260619 08:51:53.197335 26463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:51:53.197400 26463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:51:53.197456 26463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:53.198128 26463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b6c4de61634fe9b4e752c964d12767" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44801 } }
I20260619 08:51:53.198237 26463 leader_election.cc:302] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40b6c4de61634fe9b4e752c964d12767; no voters:
I20260619 08:51:53.198418 26463 leader_election.cc:288] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:51:53.198494 26468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:53.198673 26468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [term 1 LEADER]: Becoming Leader. State: Replica: 40b6c4de61634fe9b4e752c964d12767, State: Running, Role: LEADER
I20260619 08:51:53.198725 26463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:51:53.198837 26468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b6c4de61634fe9b4e752c964d12767" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44801 } }
I20260619 08:51:53.199146 26470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40b6c4de61634fe9b4e752c964d12767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b6c4de61634fe9b4e752c964d12767" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44801 } } }
I20260619 08:51:53.199215 26470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [sys.catalog]: This master's current role is: LEADER
I20260619 08:51:53.200347 26469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40b6c4de61634fe9b4e752c964d12767. Latest consensus state: current_term: 1 leader_uuid: "40b6c4de61634fe9b4e752c964d12767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b6c4de61634fe9b4e752c964d12767" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 44801 } } }
I20260619 08:51:53.200477 26469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767 [sys.catalog]: This master's current role is: LEADER
W20260619 08:51:53.200596 26484 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:51:53.200706 26484 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:51:53.200824 26475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:51:53.200999 26475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:51:53.201427 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.190670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36764 (local address 127.15.208.62:44801)
0619 08:51:53.190870 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.190879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.190912 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:51:53.191430 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.191439 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.191502 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.191679 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:53.192256 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.193660 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.194427 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.194847 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.194952 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.197320 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:53.197324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:53.197324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:53.197335 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:53.199419 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:53.199844 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:53.199846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:53.199847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:53.199930 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:53.200160 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:53.200161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:53.200161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:53.200482 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:53.201126 (+ 644us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.201192 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.201290 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260619 08:51:53.202421 26475 catalog_manager.cc:1347] Generated new cluster ID: f6a4e5ee3a5b413389db38af7795b92c
I20260619 08:51:53.202471 26475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:51:53.247803 26475 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:51:53.248378 26475 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:51:53.252506 26475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767: Generated new TSK 0
I20260619 08:51:53.252691 26475 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:51:53.280059 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44801
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:53.361590 26489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:53.361814 26489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:53.361836 26489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:53.362939 26489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:53.362973 26489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:53.362998 26489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:51:53.364471 26489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:53.364661 26489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:53.364915 26489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:53.369812 26497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:53.369788 26494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:53.369781 26495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:53.370023 26489 server_base.cc:1034] running on GCE node
I20260619 08:51:53.370218 26489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:53.370448 26489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:53.371596 26489 hybrid_clock.cc:648] HybridClock initialized: now 1781859113371577 us; error 34 us; skew 500 ppm
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859113, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:53.374035 26489 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:51:53.374500 26489 webserver.cc:460] Webserver started at http://127.15.208.1:42039/ using document root <none> and password file <none>
I20260619 08:51:53.374711 26489 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:53.374774 26489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:53.374912 26489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:53.375949 26489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "e4ae3be6aee14688a70bee1eeea4849f"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "778244beb43c8920bf25d62cd3d2084e"
server_key_iv: "7abca46608974e78343ec25831c52f20"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.376314 26489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "e4ae3be6aee14688a70bee1eeea4849f"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "778244beb43c8920bf25d62cd3d2084e"
server_key_iv: "7abca46608974e78343ec25831c52f20"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.377676 26489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:53.378362 26504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:53.378530 26489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:51:53.378623 26489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "e4ae3be6aee14688a70bee1eeea4849f"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "778244beb43c8920bf25d62cd3d2084e"
server_key_iv: "7abca46608974e78343ec25831c52f20"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.378696 26489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:53.405431 26489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:53.407629 26489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:53.407793 26489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:53.408049 26489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:53.408380 26489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:53.408438 26489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:53.408488 26489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:53.408514 26489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:53.414285 26489 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:45317
I20260619 08:51:53.414363 26617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:45317 every 8 connection(s)
I20260619 08:51:53.414700 26489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:51:53.414852 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 26489
I20260619 08:51:53.414961 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:51:53.415195 16192 external_mini_cluster.cc:1428] Setting key 5da86e949e16a30a950ffc06f9f82264
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859113, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
I20260619 08:51:53.425413 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.415547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:42787 (local address 127.15.208.62:44801)
0619 08:51:53.415604 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.415606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.415810 (+ 204us) server_negotiation.cc:407] Connection header received
0619 08:51:53.416564 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.416564 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.416575 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.416621 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:53.417890 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.418680 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.419698 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.420054 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.420133 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.422354 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:53.422356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:53.422356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:53.422364 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:53.424012 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:53.424405 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:53.424406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:53.424406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:53.424479 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:53.424702 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:53.424703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:53.424703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:53.424925 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:53.424984 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.425274 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.425345 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:51:53.425607 26620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.415525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44801 (local address 127.15.208.1:42787)
0619 08:51:53.415693 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:53.415729 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:53.416459 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:53.416692 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:53.416697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:53.417558 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:53.417801 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.417807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.418775 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.418776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.419608 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.419611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.420136 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.420137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.420236 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.420796 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:53.420810 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:53.422254 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:53.424094 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:53.424100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:53.424102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:53.424318 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:53.424550 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:53.424551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:53.424551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:53.424646 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:53.424993 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:53.424999 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:53.425155 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 08:51:53.425247 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 08:51:53.425370 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260619 08:51:53.426236 26618 heartbeater.cc:347] Connected to a master server at 127.15.208.62:44801
I20260619 08:51:53.426324 26618 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
I20260619 08:51:53.426522 26618 heartbeater.cc:511] Master 127.15.208.62:44801 requested a full tablet report, sending...
I20260619 08:51:53.427078 26427 ts_manager.cc:194] Registered new tserver with Master: e4ae3be6aee14688a70bee1eeea4849f (127.15.208.1:45317)
I20260619 08:51:53.427690 26427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:42787
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:51:53.443996 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44801
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:53.524583 26623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:53.524816 26623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:53.524839 26623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:53.526010 26623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:53.526052 26623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:53.526078 26623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:51:53.527534 26623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:53.527793 26623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:53.528016 26623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:51:53.532909 26628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:53.532903 26631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:53.532910 26629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:53.533226 26623 server_base.cc:1034] running on GCE node
I20260619 08:51:53.533406 26623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:53.533632 26623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:53.534776 26623 hybrid_clock.cc:648] HybridClock initialized: now 1781859113534758 us; error 39 us; skew 500 ppm
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859113, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:53.537322 26623 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:51:53.537796 26623 webserver.cc:460] Webserver started at http://127.15.208.2:36575/ using document root <none> and password file <none>
I20260619 08:51:53.538026 26623 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:53.538091 26623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:53.538234 26623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:53.539263 26623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "01c2a07992854e689ebf8beac1eea5f0"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "9cbf5949b3e00f7a7afd45743731e962"
server_key_iv: "f74dca9fe61566da205614fc083cc3d0"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.539619 26623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "01c2a07992854e689ebf8beac1eea5f0"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "9cbf5949b3e00f7a7afd45743731e962"
server_key_iv: "f74dca9fe61566da205614fc083cc3d0"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.541112 26623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:51:53.541833 26638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:53.542012 26623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:51:53.542110 26623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "01c2a07992854e689ebf8beac1eea5f0"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "9cbf5949b3e00f7a7afd45743731e962"
server_key_iv: "f74dca9fe61566da205614fc083cc3d0"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.542178 26623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:53.550729 26623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:53.552560 26623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:53.552711 26623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:53.552985 26623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:53.553278 26623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:53.553326 26623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:53.553365 26623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:53.553393 26623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:53.559405 26623 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:39863
I20260619 08:51:53.559456 26751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:39863 every 8 connection(s)
I20260619 08:51:53.559749 26623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859113, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:53.568594 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.560426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:41101 (local address 127.15.208.62:44801)
0619 08:51:53.560477 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.560479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.560705 (+ 226us) server_negotiation.cc:407] Connection header received
0619 08:51:53.561237 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.561238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.561247 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.561300 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:53.562166 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.562633 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.563594 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.563770 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.563824 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.565938 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:53.565940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:53.565940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:53.565948 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:53.567237 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:53.567660 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:53.567662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:53.567662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:53.567721 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:53.567953 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:53.567954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:53.567954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:53.568076 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:53.568144 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.568411 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.568493 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:51:53.568782 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.560384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44801 (local address 127.15.208.2:41101)
0619 08:51:53.560578 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:53.560619 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:53.561154 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:53.561359 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:53.561362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:53.561890 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:53.562081 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.562086 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.562706 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.562707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.563521 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.563523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.563838 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.563839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.563946 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.564453 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:53.564463 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:53.565834 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:53.567334 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:53.567341 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:53.567343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:53.567572 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:53.567788 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:53.567788 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:53.567789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:53.567889 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:53.568145 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:53.568151 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:53.568282 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 08:51:53.568379 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 08:51:53.568532 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":117,"threads_started":1}
I20260619 08:51:53.568851 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 26623
I20260619 08:51:53.568967 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:51:53.569191 16192 external_mini_cluster.cc:1428] Setting key b695736399ca255050d76f5e1d1bc348
I20260619 08:51:53.569339 26752 heartbeater.cc:347] Connected to a master server at 127.15.208.62:44801
I20260619 08:51:53.569453 26752 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:53.569638 26752 heartbeater.cc:511] Master 127.15.208.62:44801 requested a full tablet report, sending...
I20260619 08:51:53.570001 26427 ts_manager.cc:194] Registered new tserver with Master: 01c2a07992854e689ebf8beac1eea5f0 (127.15.208.2:39863)
I20260619 08:51:53.570330 26427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:41101
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:51:53.595374 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44801
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:51:53.675669 26757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:51:53.675866 26757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:51:53.675889 26757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:51:53.677050 26757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:51:53.677091 26757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:51:53.677147 26757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:51:53.678565 26757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:44801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:51:53.678797 26757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:51:53.679023 26757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:51:53.684017 26757 server_base.cc:1034] running on GCE node
W20260619 08:51:53.683951 26763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:53.683959 26765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:51:53.683990 26762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:51:53.684424 26757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:51:53.684679 26757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:51:53.685842 26757 hybrid_clock.cc:648] HybridClock initialized: now 1781859113685825 us; error 36 us; skew 500 ppm
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859113, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:51:53.688278 26757 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:51:53.688803 26757 webserver.cc:460] Webserver started at http://127.15.208.3:33223/ using document root <none> and password file <none>
I20260619 08:51:53.689047 26757 fs_manager.cc:359] Metadata directory not provided
I20260619 08:51:53.689118 26757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:51:53.689249 26757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:51:53.690305 26757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "cd2615d8394646bebaa18a7ecb860481"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "807abba66c0b11e04c024bdc7f3f3ef4"
server_key_iv: "e9e4c9680bc8e574954039c61efc922f"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.690663 26757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "cd2615d8394646bebaa18a7ecb860481"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "807abba66c0b11e04c024bdc7f3f3ef4"
server_key_iv: "e9e4c9680bc8e574954039c61efc922f"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.691928 26757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:51:53.692648 26772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:53.692853 26757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:51:53.692942 26757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "cd2615d8394646bebaa18a7ecb860481"
format_stamp: "Formatted at 2026-06-19 08:51:53 on dist-test-slave-p0h9"
server_key: "807abba66c0b11e04c024bdc7f3f3ef4"
server_key_iv: "e9e4c9680bc8e574954039c61efc922f"
server_key_version: "encryptionkey@0"
I20260619 08:51:53.693022 26757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:51:53.712337 26757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:51:53.714618 26757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:51:53.714780 26757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:51:53.715008 26757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:51:53.715313 26757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:51:53.715363 26757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:53.715405 26757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:51:53.715433 26757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:51:53.721233 26757 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:33605
I20260619 08:51:53.721278 26885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:33605 every 8 connection(s)
I20260619 08:51:53.721586 26757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859113, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:51:53.730441 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 26757
I20260619 08:51:53.730556 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:51:53.730751 16192 external_mini_cluster.cc:1428] Setting key aa50918c46213bca662861f6551514de
I20260619 08:51:53.730772 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.722348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:56987 (local address 127.15.208.62:44801)
0619 08:51:53.722409 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.722412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.722613 (+ 201us) server_negotiation.cc:407] Connection header received
0619 08:51:53.723104 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.723105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.723116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.723188 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:53.724111 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.724603 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.725582 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.725780 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.725837 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.727983 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:53.727984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:53.727984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:53.727991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:53.729435 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:53.729853 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:53.729854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:53.729854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:53.729914 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:53.730160 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:53.730161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:53.730161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:53.730275 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:53.730346 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.730606 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.730681 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:51:53.730967 26888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.722314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:44801 (local address 127.15.208.3:56987)
0619 08:51:53.722483 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:53.722524 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:53.723012 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:53.723242 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:53.723245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:53.723803 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:51:53.724023 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.724028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.724682 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.724683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.725489 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.725491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.725860 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.725861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.725978 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.726439 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:51:53.726448 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:51:53.727870 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:51:53.729536 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:53.729543 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:53.729545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:53.729778 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:53.729996 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:51:53.729997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:51:53.729997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:51:53.730096 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:51:53.730337 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:51:53.730343 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:51:53.730481 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 08:51:53.730585 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 08:51:53.730738 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260619 08:51:53.731536 26886 heartbeater.cc:347] Connected to a master server at 127.15.208.62:44801
I20260619 08:51:53.731630 26886 heartbeater.cc:464] Registering TS with master...
I20260619 08:51:53.731830 26886 heartbeater.cc:511] Master 127.15.208.62:44801 requested a full tablet report, sending...
I20260619 08:51:53.732192 26427 ts_manager.cc:194] Registered new tserver with Master: cd2615d8394646bebaa18a7ecb860481 (127.15.208.3:33605)
I20260619 08:51:53.732596 26427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:56987
I20260619 08:51:53.742893 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:51:53.750566 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.744259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36778 (local address 127.15.208.62:44801)
0619 08:51:53.744333 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.744335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.744419 (+ 84us) server_negotiation.cc:407] Connection header received
0619 08:51:53.744492 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.744493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.744503 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.744554 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:51:53.745195 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.745696 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.746470 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.746682 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.746765 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.747964 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:51:53.747966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:51:53.747966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:51:53.747975 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:51:53.749391 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:53.749749 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:53.749751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:53.749751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:53.749810 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:51:53.749992 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:51:53.749993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:51:53.749994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:51:53.750151 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:51:53.750249 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.750376 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.750456 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:51:53.751942 26427 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:51:53.753691 26427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:51:53.760805 26897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.755906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:33605 (local address 127.0.0.1:59994)
0619 08:51:53.756012 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:53.756579 (+ 567us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:53.756636 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:53.756879 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:53.756882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:53.756883 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:53.756948 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.756950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.758255 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.758256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.759598 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.759601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.760509 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.760510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.760588 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.760602 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:51:53.760649 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 08:51:53.760706 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":49,"threads_started":1}
I20260619 08:51:53.761237 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.755908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:45317 (local address 127.0.0.1:43750)
0619 08:51:53.756083 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:53.756125 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:53.756195 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:53.757211 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:53.757212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:53.757214 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:53.757257 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.757258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.758517 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.758518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.759858 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.759860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.760847 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.760848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.761087 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.761093 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:51:53.761134 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 08:51:53.761175 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260619 08:51:53.761308 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.756123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59994 (local address 127.15.208.3:33605)
0619 08:51:53.756601 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.756606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.756620 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:51:53.756706 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.756709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.756757 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.756869 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:53.757028 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.758128 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.759698 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.760399 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.761101 (+ 702us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.761164 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.761188 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.761226 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":85,"threads_started":1}
I20260619 08:51:53.761463 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.756369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43750 (local address 127.15.208.1:45317)
0619 08:51:53.757033 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.757038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.757049 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:51:53.757090 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.757092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.757108 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.757203 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:53.757347 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.758394 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.759980 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.760675 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.760756 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.760814 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.761342 (+ 528us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.761388 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":100,"threads_started":1}
I20260619 08:51:53.761495 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.756097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:39863 (local address 127.0.0.1:51856)
0619 08:51:53.756748 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:53.756757 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:53.756797 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:53.757595 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:53.757596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:53.757598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:53.757659 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.757661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.759054 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.759055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.760268 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.760271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.761243 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.761244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.761306 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.761311 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:51:53.761365 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:51:53.761412 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":67,"threads_started":1}
I20260619 08:51:53.761713 26901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.756352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51856 (local address 127.15.208.2:39863)
0619 08:51:53.757349 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.757354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.757364 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:51:53.757402 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.757405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.757421 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.757511 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:53.757755 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.758940 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.760374 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.761083 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.761153 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.761211 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.761591 (+ 380us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.761630 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":961,"thread_start_us":114,"threads_started":1}
I20260619 08:51:53.761986 26820 tablet_service.cc:1467] Processing CreateTablet for tablet 855aa73a4e124bb691d2cc8148b1eba4 (DEFAULT_TABLE table=test-table [id=0e7283de2395402bbbdec76a1360fbbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:53.762128 26552 tablet_service.cc:1467] Processing CreateTablet for tablet 855aa73a4e124bb691d2cc8148b1eba4 (DEFAULT_TABLE table=test-table [id=0e7283de2395402bbbdec76a1360fbbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:53.762367 26820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855aa73a4e124bb691d2cc8148b1eba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:53.762409 26552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855aa73a4e124bb691d2cc8148b1eba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:53.762408 26686 tablet_service.cc:1467] Processing CreateTablet for tablet 855aa73a4e124bb691d2cc8148b1eba4 (DEFAULT_TABLE table=test-table [id=0e7283de2395402bbbdec76a1360fbbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:51:53.762786 26686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855aa73a4e124bb691d2cc8148b1eba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:51:53.765563 26903 tablet_bootstrap.cc:492] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: Bootstrap starting.
I20260619 08:51:53.765851 26904 tablet_bootstrap.cc:492] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: Bootstrap starting.
I20260619 08:51:53.766342 26903 tablet_bootstrap.cc:654] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:53.766415 26904 tablet_bootstrap.cc:654] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:53.766821 26903 log.cc:824] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:53.766986 26904 log.cc:824] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:53.767283 26905 tablet_bootstrap.cc:492] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f: Bootstrap starting.
I20260619 08:51:53.767570 26903 tablet_bootstrap.cc:492] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: No bootstrap required, opened a new log
I20260619 08:51:53.767617 26903 ts_tablet_manager.cc:1397] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:51:53.767827 26905 tablet_bootstrap.cc:654] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f: Neither blocks nor log segments found. Creating new log.
I20260619 08:51:53.768308 26905 log.cc:824] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:51:53.769038 26905 tablet_bootstrap.cc:492] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f: No bootstrap required, opened a new log
I20260619 08:51:53.769129 26905 ts_tablet_manager.cc:1397] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:51:53.769286 26904 tablet_bootstrap.cc:492] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: No bootstrap required, opened a new log
I20260619 08:51:53.769191 26903 raft_consensus.cc:348] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 } } peers { permanent_uuid: "cd2615d8394646bebaa18a7ecb860481" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 33605 } } peers { permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 } }
I20260619 08:51:53.769343 26903 raft_consensus.cc:374] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:53.769354 26904 ts_tablet_manager.cc:1397] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 08:51:53.769389 26903 raft_consensus.cc:729] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01c2a07992854e689ebf8beac1eea5f0, State: Initialized, Role: FOLLOWER
I20260619 08:51:53.769526 26903 consensus_queue.cc:260] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 } } peers { permanent_uuid: "cd2615d8394646bebaa18a7ecb860481" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 33605 } } peers { permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 } }
I20260619 08:51:53.769791 26903 ts_tablet_manager.cc:1428] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:51:53.769950 26752 heartbeater.cc:503] Master 127.15.208.62:44801 was elected leader, sending a full tablet report...
I20260619 08:51:53.770958 26904 raft_consensus.cc:348] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 } } peers { permanent_uuid: "cd2615d8394646bebaa18a7ecb860481" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 33605 } } peers { permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 } }
I20260619 08:51:53.771076 26904 raft_consensus.cc:374] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:53.771096 26904 raft_consensus.cc:729] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2615d8394646bebaa18a7ecb860481, State: Initialized, Role: FOLLOWER
I20260619 08:51:53.771004 26905 raft_consensus.cc:348] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 } } peers { permanent_uuid: "cd2615d8394646bebaa18a7ecb860481" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 33605 } } peers { permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 } }
I20260619 08:51:53.771157 26905 raft_consensus.cc:374] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:51:53.771198 26905 raft_consensus.cc:729] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4ae3be6aee14688a70bee1eeea4849f, State: Initialized, Role: FOLLOWER
I20260619 08:51:53.771199 26904 consensus_queue.cc:260] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 } } peers { permanent_uuid: "cd2615d8394646bebaa18a7ecb860481" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 33605 } } peers { permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 } }
I20260619 08:51:53.771317 26905 consensus_queue.cc:260] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 } } peers { permanent_uuid: "cd2615d8394646bebaa18a7ecb860481" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 33605 } } peers { permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 } }
I20260619 08:51:53.771457 26904 ts_tablet_manager.cc:1428] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:51:53.771457 26886 heartbeater.cc:503] Master 127.15.208.62:44801 was elected leader, sending a full tablet report...
I20260619 08:51:53.771601 26905 ts_tablet_manager.cc:1428] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:51:53.771724 26618 heartbeater.cc:503] Master 127.15.208.62:44801 was elected leader, sending a full tablet report...
I20260619 08:51:53.802911 26910 raft_consensus.cc:482] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:51:53.803047 26910 raft_consensus.cc:504] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 } } peers { permanent_uuid: "cd2615d8394646bebaa18a7ecb860481" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 33605 } } peers { permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 } }
I20260619 08:51:53.803352 26910 leader_election.cc:288] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4ae3be6aee14688a70bee1eeea4849f (127.15.208.1:45317), 01c2a07992854e689ebf8beac1eea5f0 (127.15.208.2:39863)
I20260619 08:51:53.806247 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.803679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:55831 (local address 127.15.208.1:45317)
0619 08:51:53.803757 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.803759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.803767 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:53.803794 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.803795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.803804 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.803851 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:53.804051 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.804566 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.805528 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.805903 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.805990 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.806005 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.806148 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.806184 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 08:51:53.806247 26888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.803599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:45317 (local address 127.15.208.3:55831)
0619 08:51:53.803662 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:53.803676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:53.803727 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:53.803881 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:53.803882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:53.803884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:53.803971 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.803973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.804651 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.804652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.805412 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.805414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.805988 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.805988 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.806067 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.806070 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:51:53.806130 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 08:51:53.806173 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 08:51:53.806533 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.803719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:39863 (local address 127.15.208.3:36751)
0619 08:51:53.803945 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:53.803953 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:51:53.804030 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:51:53.804166 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:51:53.804166 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:51:53.804168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:51:53.804245 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.804252 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.804926 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.804927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.805655 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:51:53.805657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.806198 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.806199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:51:53.806277 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.806285 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 08:51:53.806407 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 08:51:53.806454 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":111,"threads_started":1}
I20260619 08:51:53.806591 26901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.803794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:36751 (local address 127.15.208.2:39863)
0619 08:51:53.803859 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.803861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.804008 (+ 147us) server_negotiation.cc:407] Connection header received
0619 08:51:53.804082 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.804083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.804093 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.804161 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:51:53.804314 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.804850 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.805741 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.806118 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.806198 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.806205 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.806472 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.806518 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:51:53.806609 26572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855aa73a4e124bb691d2cc8148b1eba4" candidate_uuid: "cd2615d8394646bebaa18a7ecb860481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4ae3be6aee14688a70bee1eeea4849f" is_pre_election: true
I20260619 08:51:53.806777 26572 raft_consensus.cc:2454] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd2615d8394646bebaa18a7ecb860481 in term 0.
I20260619 08:51:53.806872 26706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855aa73a4e124bb691d2cc8148b1eba4" candidate_uuid: "cd2615d8394646bebaa18a7ecb860481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01c2a07992854e689ebf8beac1eea5f0" is_pre_election: true
I20260619 08:51:53.806973 26776 leader_election.cc:302] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd2615d8394646bebaa18a7ecb860481, e4ae3be6aee14688a70bee1eeea4849f; no voters:
I20260619 08:51:53.807026 26706 raft_consensus.cc:2454] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd2615d8394646bebaa18a7ecb860481 in term 0.
I20260619 08:51:53.807103 26910 raft_consensus.cc:2781] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:51:53.807185 26910 raft_consensus.cc:482] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:51:53.807220 26910 raft_consensus.cc:3037] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:53.807898 26910 raft_consensus.cc:504] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 } } peers { permanent_uuid: "cd2615d8394646bebaa18a7ecb860481" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 33605 } } peers { permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 } }
I20260619 08:51:53.808043 26910 leader_election.cc:288] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [CANDIDATE]: Term 1 election: Requested vote from peers e4ae3be6aee14688a70bee1eeea4849f (127.15.208.1:45317), 01c2a07992854e689ebf8beac1eea5f0 (127.15.208.2:39863)
I20260619 08:51:53.808193 26572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855aa73a4e124bb691d2cc8148b1eba4" candidate_uuid: "cd2615d8394646bebaa18a7ecb860481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4ae3be6aee14688a70bee1eeea4849f"
I20260619 08:51:53.808238 26706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855aa73a4e124bb691d2cc8148b1eba4" candidate_uuid: "cd2615d8394646bebaa18a7ecb860481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01c2a07992854e689ebf8beac1eea5f0"
I20260619 08:51:53.808291 26572 raft_consensus.cc:3037] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:53.808303 26706 raft_consensus.cc:3037] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:51:53.809078 26706 raft_consensus.cc:2454] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd2615d8394646bebaa18a7ecb860481 in term 1.
I20260619 08:51:53.809144 26572 raft_consensus.cc:2454] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd2615d8394646bebaa18a7ecb860481 in term 1.
I20260619 08:51:53.809262 26776 leader_election.cc:302] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01c2a07992854e689ebf8beac1eea5f0, cd2615d8394646bebaa18a7ecb860481; no voters:
I20260619 08:51:53.809388 26910 raft_consensus.cc:2781] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:51:53.809571 26910 raft_consensus.cc:686] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 1 LEADER]: Becoming Leader. State: Replica: cd2615d8394646bebaa18a7ecb860481, State: Running, Role: LEADER
I20260619 08:51:53.809670 26910 consensus_queue.cc:237] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 } } peers { permanent_uuid: "cd2615d8394646bebaa18a7ecb860481" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 33605 } } peers { permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 } }
W20260619 08:51:53.810273 26753 tablet.cc:2369] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:51:53.810534 26427 catalog_manager.cc:5495] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd2615d8394646bebaa18a7ecb860481 (127.15.208.3). New cstate: current_term: 1 leader_uuid: "cd2615d8394646bebaa18a7ecb860481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd2615d8394646bebaa18a7ecb860481" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 33605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 } health_report { overall_health: UNKNOWN } } }
I20260619 08:51:53.816862 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.814478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.15.208.3:33605)
0619 08:51:53.814526 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:51:53.814528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:51:53.814536 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:51:53.814604 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:51:53.814605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:51:53.814620 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:51:53.814698 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:51:53.814840 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.815297 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.816070 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:51:53.816260 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:51:53.816342 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:51:53.816513 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:51:53.816611 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:51:53.816705 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:51:53.816723 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 08:51:53.816768 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 08:51:53.818470 26572 raft_consensus.cc:1261] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f [term 1 FOLLOWER]: Refusing update from remote peer cd2615d8394646bebaa18a7ecb860481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:53.818490 26706 raft_consensus.cc:1261] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0 [term 1 FOLLOWER]: Refusing update from remote peer cd2615d8394646bebaa18a7ecb860481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:51:53.818805 26910 consensus_queue.cc:1035] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01c2a07992854e689ebf8beac1eea5f0" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:53.818956 26913 consensus_queue.cc:1035] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4ae3be6aee14688a70bee1eeea4849f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:51:53.821403 26918 mvcc.cc:205] Tried to move back new op lower bound from 7298494930198200320 to 7298494930165133312. Current Snapshot: MvccSnapshot[applied={T|T < 7298494930198200320}]
I20260619 08:51:53.821406 26917 mvcc.cc:205] Tried to move back new op lower bound from 7298494930198200320 to 7298494930165133312. Current Snapshot: MvccSnapshot[applied={T|T < 7298494930198200320}]
I20260619 08:51:53.823457 26425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:53.823571 26425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:51:53.824273 26425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40b6c4de61634fe9b4e752c964d12767: Sending DeleteTablet for 3 replicas of tablet 855aa73a4e124bb691d2cc8148b1eba4
I20260619 08:51:53.824601 26820 tablet_service.cc:1514] Processing DeleteTablet for tablet 855aa73a4e124bb691d2cc8148b1eba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:53 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:59994
I20260619 08:51:53.824642 26552 tablet_service.cc:1514] Processing DeleteTablet for tablet 855aa73a4e124bb691d2cc8148b1eba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:53 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:43750
I20260619 08:51:53.824689 26686 tablet_service.cc:1514] Processing DeleteTablet for tablet 855aa73a4e124bb691d2cc8148b1eba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:51:53 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:51856
I20260619 08:51:53.824867 26927 tablet_replica.cc:330] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f: stopping tablet replica
I20260619 08:51:53.824889 26926 tablet_replica.cc:330] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: stopping tablet replica
I20260619 08:51:53.824970 26926 raft_consensus.cc:2229] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:51:53.824968 26927 raft_consensus.cc:2229] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:53.825062 26927 raft_consensus.cc:2258] T 855aa73a4e124bb691d2cc8148b1eba4 P e4ae3be6aee14688a70bee1eeea4849f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:53.825069 26926 raft_consensus.cc:2258] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:53.825177 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 26489
I20260619 08:51:53.826216 26926 ts_tablet_manager.cc:1905] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:53.828239 26926 ts_tablet_manager.cc:1918] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:53.828303 26926 log.cc:1196] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/855aa73a4e124bb691d2cc8148b1eba4
I20260619 08:51:53.828572 26926 ts_tablet_manager.cc:1939] T 855aa73a4e124bb691d2cc8148b1eba4 P cd2615d8394646bebaa18a7ecb860481: Deleting consensus metadata
I20260619 08:51:53.828999 26412 catalog_manager.cc:4841] TS cd2615d8394646bebaa18a7ecb860481 (127.15.208.3:33605): tablet 855aa73a4e124bb691d2cc8148b1eba4 (table test-table [id=0e7283de2395402bbbdec76a1360fbbd]) successfully deleted
I20260619 08:51:53.829682 26928 tablet_replica.cc:330] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: stopping tablet replica
I20260619 08:51:53.829789 26928 raft_consensus.cc:2229] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:51:53.829896 26928 raft_consensus.cc:2258] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:51:53.830335 26928 ts_tablet_manager.cc:1905] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:51:53.831866 26928 ts_tablet_manager.cc:1918] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:51:53.831939 26928 log.cc:1196] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/855aa73a4e124bb691d2cc8148b1eba4
I20260619 08:51:53.832204 26928 ts_tablet_manager.cc:1939] T 855aa73a4e124bb691d2cc8148b1eba4 P 01c2a07992854e689ebf8beac1eea5f0: Deleting consensus metadata
W20260619 08:51:53.832409 26414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:45317 (error 108)
I20260619 08:51:53.832633 26414 catalog_manager.cc:4841] TS 01c2a07992854e689ebf8beac1eea5f0 (127.15.208.2:39863): tablet 855aa73a4e124bb691d2cc8148b1eba4 (table test-table [id=0e7283de2395402bbbdec76a1360fbbd]) successfully deleted
I20260619 08:51:53.832712 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 26623
I20260619 08:51:53.833350 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:51:53.832801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:45317 (local address 127.0.0.1:43764)
0619 08:51:53.833271 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0619 08:51:53.833312 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450}
W20260619 08:51:53.833940 26414 catalog_manager.cc:4584] TS e4ae3be6aee14688a70bee1eeea4849f (127.15.208.1:45317): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 855aa73a4e124bb691d2cc8148b1eba4: Network error: Client connection negotiation failed: client connection to 127.15.208.1:45317: connect: Connection refused (error 111)
I20260619 08:51:53.838264 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 26757
I20260619 08:51:53.843040 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 26395
2026-06-19T08:51:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1937 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26932](info): set up 2 sockets
Jun 19 08:51:53 dist-test-slave-p0h9 krb5kdc[26932](info): commencing operation
krb5kdc: starting...
W20260619 08:51:54.874199 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:51:54 dist-test-slave-p0h9 krb5kdc[26932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:51:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:51:54Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/rangeradmin_127.15.208.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/rangeradmin_127.15.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/rangerlookup_127.15.208.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/rangerlookup_127.15.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/HTTP_127.15.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/HTTP_127.15.208.20@KRBTEST.COM.keytab.
I20260619 08:51:55.036545 16192 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLqpuKl/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/postgres -l logfile start
2026-06-19 08:51:57.433 UTC [26962] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 08:51:57.433 UTC [26962] LOG: listening on IPv4 address "127.15.208.20", port 49599
2026-06-19 08:51:57.436 UTC [26962] LOG: listening on Unix socket "/tmp/.s.PGSQL.49599"
2026-06-19 08:51:57.462 UTC [26965] LOG: database system was shut down at 2026-06-19 08:51:56 UTC
2026-06-19 08:51:57.466 UTC [26962] LOG: database system is ready to accept connections
I20260619 08:51:58.434617 16192 mini_postgres.cc:96] Postgres bound to 49599
2026-06-19 08:51:58.440 UTC [26973] FATAL: database "slave" does not exist
127.15.208.20:49599 - accepting connections
I20260619 08:51:58.441246 16192 mini_ranger.cc:161] Starting Ranger...
I20260619 08:51:58.455742 16192 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 08:51:58.938858 16192 mini_ranger.cc:87] Created ranger Postgres database
I20260619 08:51:58.938946 16192 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-admin
2026-06-19 08:51:59,265 [I] DB FLAVOR :POSTGRES
2026-06-19 08:51:59,265 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:51:59,265 [I] Checking connection..
2026-06-19 08:51:59,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 08:51:59,540 [I] Checking connection passed.
2026-06-19 08:51:59,540 [I] --------- Verifying version history table ---------
2026-06-19 08:51:59,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:51:59,772 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:51:59,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:52:00,018 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:52:00,018 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 08:52:00,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 08:52:00.271 UTC [27037] WARNING: there is no transaction in progress
2026-06-19 08:52:00,277 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 08:52:00,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:52:00,522 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 08:52:00,522 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:52:00,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 08:52:00,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 08:52:01,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p0h9.c.gcp-upstream.internal','N') ;"
2026-06-19 08:52:01,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:52:01,497 [I] Table x_portal_user does not exist in database ranger
2026-06-19 08:52:01,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:52:01,758 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 08:52:01,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:52:02,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:52:02,261 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 08:52:02,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 08:52:03.240 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.256 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.272 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.288 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.304 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.320 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.381 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.387 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.402 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.412 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.421 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.435 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.444 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.451 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.457 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:03.463 UTC [27145] WARNING: there is no transaction in progress
2026-06-19 08:52:04,061 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 08:52:04,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 08:52:04,281 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 08:52:04,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:52:04,525 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 08:52:04,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:52:04,793 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 08:52:04,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:52:05,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:52:05,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p0h9.c.gcp-upstream.internal';"
2026-06-19 08:52:05,511 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 08:52:05,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 08:52:05,771 [I] DB_PATCHES have already been applied
I20260619 08:52:05.777801 16192 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLqpuKl/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 08:52:05.777868 16192 mini_ranger.cc:193] Using host: 127.15.208.20
I20260619 08:52:05.781026 16192 mini_ranger.cc:227] Ranger admin URL: http://127.15.208.20:35793
Jun 19, 2026 8:52:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:52:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:52:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:52:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:52:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:52:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:52:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 08:52:06.913370 16192 mini_ranger.cc:160] Time spent starting Ranger: real 8.472s user 0.000s sys 0.010s
Jun 19, 2026 8:52:26 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:52:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 8:52:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:52:26 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:52:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:52:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:52:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:52:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:52:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:52:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:52:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:52:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:52:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:52:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:52:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:52:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:52:27 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:52:27 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 08:52:28.547322 16192 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/rangerkms_127.15.208.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/rangerkms_127.15.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.208.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.208.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.208.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/HTTP_127.15.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.208.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/HTTP_127.15.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 08:52:28 dist-test-slave-p0h9 krb5kdc[26932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859148, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 08:52:28.627099 16192 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 08:52:28.636597 16192 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 08:52:29.419498 16192 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 08:52:29.419600 16192 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-kms
I20260619 08:52:29.419623 16192 mini_ranger_kms.cc:227] Using postgres at 127.15.208.20:49599
2026-06-19 08:52:29,627 [I] DB FLAVOR :POSTGRES
2026-06-19 08:52:29,628 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:52:29,628 [I] Checking connection
2026-06-19 08:52:29,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 08:52:29,869 [I] connection success
2026-06-19 08:52:29,869 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 08:52:29,869 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 08:52:29,870 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 08:52:30,104 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 08:52:30,104 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:52:30,104 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 08:52:30,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:49599/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 08:52:30,349 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 08:52:30.823271 16192 mini_ranger_kms.cc:310] Created kms service
I20260619 08:52:31.020412 16192 mini_ranger_kms.cc:326] Created kudu user
I20260619 08:52:31.063851 16192 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 08:52:31 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859148, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:52:31.463263 16192 mini_ranger_kms.cc:384] Added ranger policy
I20260619 08:52:31.463383 16192 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 08:52:31.463407 16192 mini_ranger_kms.cc:240] Using host: 127.15.208.20
I20260619 08:52:31.466922 16192 mini_ranger_kms.cc:276] Ranger KMS PID: 27325
I20260619 08:52:31.467064 16192 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.208.20:60649
Jun 19, 2026 8:52:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:52:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:52:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:52:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:52:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:52:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 19 08:52:33 dist-test-slave-p0h9 krb5kdc[26932](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781859153, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.208.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:52:33.726082 16192 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.099s user 0.000s sys 0.191s
I20260619 08:52:33.726198 16192 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 08:52:33.726289 16192 mini_ranger_kms.cc:401] 127.15.208.20:60649/kms/v1/keys
Jun 19 08:52:33 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859153, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.208.20@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
Jun 19 08:52:34 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859153, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.208.20@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
Jun 19 08:52:36 dist-test-slave-p0h9 krb5kdc[26932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:52:36.593295 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:37413
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:37413
--ranger_config_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60649/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:52:36.678745 27376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:52:36.678973 27376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:52:36.678997 27376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:52:36.680197 27376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:52:36.680246 27376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:52:36.680263 27376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:52:36.680277 27376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:52:36.680291 27376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:52:36.681919 27376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.208.20:60649/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:37413
--ranger_config_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:37413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:52:36.682160 27376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:52:36.682442 27376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:52:36.687633 27381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:52:36.687659 27384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:52:36.687745 27376 server_base.cc:1034] running on GCE node
W20260619 08:52:36.687789 27382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:52:36.688256 27376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:52:36.688630 27376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:52:36.689810 27376 hybrid_clock.cc:648] HybridClock initialized: now 1781859156689793 us; error 49 us; skew 500 ppm
Jun 19 08:52:36 dist-test-slave-p0h9 krb5kdc[26932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859156, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:52:36.695538 27376 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:52:36.696074 27376 webserver.cc:460] Webserver started at http://127.15.208.62:41175/ using document root <none> and password file <none>
I20260619 08:52:36.696321 27376 fs_manager.cc:359] Metadata directory not provided
I20260619 08:52:36.696369 27376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:52:36.696491 27376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:52:36 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859156, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:52:36.742539 27376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "843928fac50046b6971464ffc6012868"
format_stamp: "Formatted at 2026-06-19 08:52:36 on dist-test-slave-p0h9"
server_key: "e674358b925a5b09f6174ab6b2187724"
server_key_iv: "36175e4463c011164dfdef3805551224"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:36.743010 27376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "843928fac50046b6971464ffc6012868"
format_stamp: "Formatted at 2026-06-19 08:52:36 on dist-test-slave-p0h9"
server_key: "e674358b925a5b09f6174ab6b2187724"
server_key_iv: "36175e4463c011164dfdef3805551224"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:36.744798 27376 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260619 08:52:36.759817 27391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:52:36.759994 27376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:52:36.760080 27376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "843928fac50046b6971464ffc6012868"
format_stamp: "Formatted at 2026-06-19 08:52:36 on dist-test-slave-p0h9"
server_key: "e674358b925a5b09f6174ab6b2187724"
server_key_iv: "36175e4463c011164dfdef3805551224"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:36.760174 27376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:52:36.784979 27376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:52:36.787020 27376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:52:36.787174 27376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:52:36.792308 27376 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:37413
I20260619 08:52:36.792384 27443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:37413 every 8 connection(s)
I20260619 08:52:36.792784 27376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:52:36.793488 27444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:52:36.795775 27444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868: Bootstrap starting.
I20260619 08:52:36.796340 27444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868: Neither blocks nor log segments found. Creating new log.
I20260619 08:52:36.796636 27444 log.cc:824] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868: Log is configured to *not* fsync() on all Append() calls
I20260619 08:52:36.797462 27444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868: No bootstrap required, opened a new log
I20260619 08:52:36.799490 27444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843928fac50046b6971464ffc6012868" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37413 } }
I20260619 08:52:36.799671 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 27376
I20260619 08:52:36.799700 27444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:52:36.799741 27444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843928fac50046b6971464ffc6012868, State: Initialized, Role: FOLLOWER
I20260619 08:52:36.799772 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:52:36.799974 16192 external_mini_cluster.cc:1428] Setting key cc5e1fa1b8707123dc3d609c98325d0e
I20260619 08:52:36.799934 27444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843928fac50046b6971464ffc6012868" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37413 } }
I20260619 08:52:36.800019 27444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:52:36.800071 27444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:52:36.800144 27444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:52:36.800916 27444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843928fac50046b6971464ffc6012868" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37413 } }
I20260619 08:52:36.801019 27444 leader_election.cc:302] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 843928fac50046b6971464ffc6012868; no voters:
I20260619 08:52:36.801203 27444 leader_election.cc:288] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:52:36.801389 27449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:52:36.801524 27449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [term 1 LEADER]: Becoming Leader. State: Replica: 843928fac50046b6971464ffc6012868, State: Running, Role: LEADER
I20260619 08:52:36.801625 27449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843928fac50046b6971464ffc6012868" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37413 } }
I20260619 08:52:36.801806 27444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:52:36.801908 27449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "843928fac50046b6971464ffc6012868" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843928fac50046b6971464ffc6012868" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37413 } } }
I20260619 08:52:36.801993 27449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [sys.catalog]: This master's current role is: LEADER
I20260619 08:52:36.802155 27449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 843928fac50046b6971464ffc6012868. Latest consensus state: current_term: 1 leader_uuid: "843928fac50046b6971464ffc6012868" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843928fac50046b6971464ffc6012868" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37413 } } }
I20260619 08:52:36.802232 27449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868 [sys.catalog]: This master's current role is: LEADER
I20260619 08:52:36.802454 27453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:52:36.804119 27453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:52:36.804397 27444 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Jun 19 08:52:36 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:52:36.808120 27453 catalog_manager.cc:1347] Generated new cluster ID: 7633ea60cede44b9b92b66a35ce37999
I20260619 08:52:36.810024 27453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:52:36.815891 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:36.800631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.15.208.62:37413)
0619 08:52:36.800849 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:36.800859 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:36.800891 (+ 32us) server_negotiation.cc:407] Connection header received
0619 08:52:36.801004 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:36.801011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:36.801079 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:36.801223 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:52:36.802089 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:36.803527 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:36.804545 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:36.805134 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:36.805212 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:36.808074 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:52:36.808078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:52:36.808078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:52:36.808090 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:52:36.812790 (+ 4700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:52:36.813335 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:52:36.813338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:52:36.813339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:52:36.814060 (+ 721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:52:36.814305 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:52:36.814306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:52:36.814306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:52:36.814793 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:52:36.815365 (+ 572us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:36.815447 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 08:52:36.815623 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":72,"threads_started":1}
I20260619 08:52:36.825415 27453 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:52:36.826233 27453 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:52:36.846903 27453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868: Generated new TSK 0
I20260619 08:52:36.847178 27453 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:52:37.688568 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60649/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37413
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:52:37.779860 27489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:52:37.780097 27489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:52:37.780138 27489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:52:37.781430 27489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:52:37.781519 27489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:52:37.781579 27489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:52:37.783069 27489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.208.20:60649/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:52:37.783352 27489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:52:37.783601 27489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:52:37.789539 27495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:52:37.789601 27497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:52:37.789515 27494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:52:37.789927 27489 server_base.cc:1034] running on GCE node
I20260619 08:52:37.790125 27489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:52:37.790380 27489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:52:37.791561 27489 hybrid_clock.cc:648] HybridClock initialized: now 1781859157791560 us; error 53 us; skew 500 ppm
Jun 19 08:52:37 dist-test-slave-p0h9 krb5kdc[26932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859157, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:52:37.794157 27489 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:52:37.794649 27489 webserver.cc:460] Webserver started at http://127.15.208.1:40705/ using document root <none> and password file <none>
I20260619 08:52:37.794896 27489 fs_manager.cc:359] Metadata directory not provided
I20260619 08:52:37.794965 27489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:52:37.795116 27489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:52:37 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859157, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:52:37.809330 27489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "5e9dac66343c4706a8d5d2fb901ba6ed"
format_stamp: "Formatted at 2026-06-19 08:52:37 on dist-test-slave-p0h9"
server_key: "9ff4b48bea2a798a6d014c8ec8987f02"
server_key_iv: "2e47c2f599bac1740492849829aebf41"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:37.809772 27489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "5e9dac66343c4706a8d5d2fb901ba6ed"
format_stamp: "Formatted at 2026-06-19 08:52:37 on dist-test-slave-p0h9"
server_key: "9ff4b48bea2a798a6d014c8ec8987f02"
server_key_iv: "2e47c2f599bac1740492849829aebf41"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:37.813591 27489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260619 08:52:37.832664 27504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:52:37.833022 27489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:52:37.833141 27489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "5e9dac66343c4706a8d5d2fb901ba6ed"
format_stamp: "Formatted at 2026-06-19 08:52:37 on dist-test-slave-p0h9"
server_key: "9ff4b48bea2a798a6d014c8ec8987f02"
server_key_iv: "2e47c2f599bac1740492849829aebf41"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:37.833251 27489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:52:37.857996 27489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:52:37.860491 27489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:52:37.860633 27489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:52:37.860924 27489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:52:37.861284 27489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:52:37.861327 27489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:52:37.861358 27489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:52:37.861385 27489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:52:37.870821 27489 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:41345
I20260619 08:52:37.871409 27489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:52:37.871608 27619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:41345 every 8 connection(s)
I20260619 08:52:37.875887 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 27489
I20260619 08:52:37.876017 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:52:37.876199 16192 external_mini_cluster.cc:1428] Setting key b5de9ea1c00053a0472b66a4e2b25528
Jun 19 08:52:37 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859157, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
I20260619 08:52:37.886766 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:37.872339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:37413 (local address 127.15.208.1:50639)
0619 08:52:37.872507 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:52:37.872550 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:52:37.873172 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:52:37.873479 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:52:37.873483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:52:37.873688 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:52:37.873884 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:37.873888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:37.874645 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:37.874646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:37.875458 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:37.875461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:37.876004 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:37.876005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:37.876156 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:37.876821 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:52:37.876831 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:52:37.878579 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:52:37.884999 (+ 6420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:52:37.885009 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:52:37.885011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:52:37.885354 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:52:37.885621 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:52:37.885621 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:52:37.885622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:52:37.885745 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:52:37.886076 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:52:37.886084 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:52:37.886277 (+ 193us) client_negotiation.cc:769] Sending connection context
0619 08:52:37.886423 (+ 146us) client_negotiation.cc:240] Negotiation successful
0619 08:52:37.886590 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260619 08:52:37.887317 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:37.872351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:50639 (local address 127.15.208.62:37413)
0619 08:52:37.872717 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:37.872719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:37.872763 (+ 44us) server_negotiation.cc:407] Connection header received
0619 08:52:37.873306 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:37.873307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:37.873325 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:37.873415 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:52:37.874003 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:37.874537 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:37.875570 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:37.875889 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:37.875968 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:37.878670 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:52:37.878672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:52:37.878672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:52:37.878681 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:52:37.881515 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:52:37.885467 (+ 3952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:52:37.885468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:52:37.885468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:52:37.885539 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:52:37.885831 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:52:37.885832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:52:37.885832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:52:37.885998 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:52:37.886086 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:37.887083 (+ 997us) server_negotiation.cc:299] Negotiation successful
0619 08:52:37.887187 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":64,"threads_started":1}
I20260619 08:52:37.887748 27620 heartbeater.cc:347] Connected to a master server at 127.15.208.62:37413
I20260619 08:52:37.887856 27620 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
I20260619 08:52:37.888074 27620 heartbeater.cc:511] Master 127.15.208.62:37413 requested a full tablet report, sending...
I20260619 08:52:37.888653 27408 ts_manager.cc:194] Registered new tserver with Master: 5e9dac66343c4706a8d5d2fb901ba6ed (127.15.208.1:41345)
I20260619 08:52:37.889577 27408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:50639
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:52:37.907023 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60649/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37413
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:52:38.013116 27626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:52:38.013334 27626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:52:38.013372 27626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:52:38.014520 27626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:52:38.014591 27626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:52:38.014637 27626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:52:38.016041 27626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.208.20:60649/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:52:38.016291 27626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:52:38.016499 27626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:52:38.021358 27636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:52:38.021656 27633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:52:38.021852 27634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:52:38.022215 27626 server_base.cc:1034] running on GCE node
I20260619 08:52:38.022439 27626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:52:38.022643 27626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:52:38.023772 27626 hybrid_clock.cc:648] HybridClock initialized: now 1781859158023744 us; error 41 us; skew 500 ppm
Jun 19 08:52:38 dist-test-slave-p0h9 krb5kdc[26932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859158, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:52:38.026367 27626 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:52:38.026847 27626 webserver.cc:460] Webserver started at http://127.15.208.2:36357/ using document root <none> and password file <none>
I20260619 08:52:38.027086 27626 fs_manager.cc:359] Metadata directory not provided
I20260619 08:52:38.027151 27626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:52:38.027287 27626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:52:38 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859158, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:52:38.037823 27626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "c62c659a276947ad8fb69afa1acb028e"
format_stamp: "Formatted at 2026-06-19 08:52:38 on dist-test-slave-p0h9"
server_key: "7a60a9dfedab00ae574b105d657a0dbf"
server_key_iv: "2e4b5cfc36f90b416bd84e3688e10af4"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:38.038247 27626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "c62c659a276947ad8fb69afa1acb028e"
format_stamp: "Formatted at 2026-06-19 08:52:38 on dist-test-slave-p0h9"
server_key: "7a60a9dfedab00ae574b105d657a0dbf"
server_key_iv: "2e4b5cfc36f90b416bd84e3688e10af4"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:38.044224 27626 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.000s
I20260619 08:52:38.063357 27643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
Jun 19 08:52:38 dist-test-slave-p0h9 krb5kdc[26932](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781859158, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:52:38.063669 27626 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260619 08:52:38.063796 27626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "c62c659a276947ad8fb69afa1acb028e"
format_stamp: "Formatted at 2026-06-19 08:52:38 on dist-test-slave-p0h9"
server_key: "7a60a9dfedab00ae574b105d657a0dbf"
server_key_iv: "2e4b5cfc36f90b416bd84e3688e10af4"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:38.063898 27626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:52:38.110128 27626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:52:38.112702 27626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:52:38.112895 27626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:52:38.113158 27626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:52:38.113528 27626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:52:38.113574 27626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:52:38.113610 27626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:52:38.113631 27626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:52:38.123981 27626 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:39707
I20260619 08:52:38.124401 27626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:52:38.125021 27756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:39707 every 8 connection(s)
Jun 19 08:52:38 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859158, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:52:38.133561 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 27626
I20260619 08:52:38.133678 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:52:38.133873 16192 external_mini_cluster.cc:1428] Setting key 504a83f5c7812a847d613a774f502795
I20260619 08:52:38.134078 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.125556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:50653 (local address 127.15.208.62:37413)
0619 08:52:38.125633 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:38.125636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:38.125651 (+ 15us) server_negotiation.cc:407] Connection header received
0619 08:52:38.126060 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:38.126061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:38.126076 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:38.126133 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:52:38.126707 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.127238 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.128396 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.128623 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.128711 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.130934 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:52:38.130936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:52:38.130937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:52:38.130945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:52:38.132471 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:52:38.132977 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:52:38.132978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:52:38.132979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:52:38.133052 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:52:38.133309 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:52:38.133310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:52:38.133311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:52:38.133430 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:52:38.133510 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:38.133819 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 08:52:38.133976 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:52:38.134178 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.125184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:37413 (local address 127.15.208.2:50653)
0619 08:52:38.125353 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 08:52:38.125396 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:52:38.125935 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:52:38.126215 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:52:38.126219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:52:38.126406 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:52:38.126605 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.126610 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.127381 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.127383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.128301 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.128303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.128744 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.128745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.128870 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.129420 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:52:38.129440 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:52:38.130819 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:52:38.132573 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:52:38.132580 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:52:38.132583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:52:38.132890 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:52:38.133117 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:52:38.133118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:52:38.133118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:52:38.133241 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:52:38.133502 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:52:38.133509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:52:38.133682 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 08:52:38.133790 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 08:52:38.133943 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260619 08:52:38.134774 27757 heartbeater.cc:347] Connected to a master server at 127.15.208.62:37413
I20260619 08:52:38.134879 27757 heartbeater.cc:464] Registering TS with master...
I20260619 08:52:38.135073 27757 heartbeater.cc:511] Master 127.15.208.62:37413 requested a full tablet report, sending...
I20260619 08:52:38.135458 27408 ts_manager.cc:194] Registered new tserver with Master: c62c659a276947ad8fb69afa1acb028e (127.15.208.2:39707)
I20260619 08:52:38.135821 27408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:50653
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:52:38.168980 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60649/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37413
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:41593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:52:38.296084 27763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:52:38.296384 27763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:52:38.296428 27763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:52:38.297849 27763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:52:38.297936 27763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:52:38.297984 27763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:52:38.300396 27763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:41593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.208.20:60649/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:52:38.300644 27763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:52:38.300948 27763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:52:38.313827 27768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:52:38.313839 27771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:52:38.314141 27769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:52:38.317884 27763 server_base.cc:1034] running on GCE node
I20260619 08:52:38.318128 27763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:52:38.318388 27763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:52:38.319550 27763 hybrid_clock.cc:648] HybridClock initialized: now 1781859158319532 us; error 37 us; skew 500 ppm
Jun 19 08:52:38 dist-test-slave-p0h9 krb5kdc[26932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859158, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:52:38.322715 27763 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:52:38.323246 27763 webserver.cc:460] Webserver started at http://127.15.208.3:41445/ using document root <none> and password file <none>
I20260619 08:52:38.323503 27763 fs_manager.cc:359] Metadata directory not provided
I20260619 08:52:38.323560 27763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:52:38.323683 27763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:52:38 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859158, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:52:38.343212 27763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "b33042cd01cd458bbcf6c80907f70efa"
format_stamp: "Formatted at 2026-06-19 08:52:38 on dist-test-slave-p0h9"
server_key: "50d13e114ed5b0c3d0d74d8b9513a24c"
server_key_iv: "163b0fa7a697c2a12f0a273b1bf8ecf3"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:38.343712 27763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "b33042cd01cd458bbcf6c80907f70efa"
format_stamp: "Formatted at 2026-06-19 08:52:38 on dist-test-slave-p0h9"
server_key: "50d13e114ed5b0c3d0d74d8b9513a24c"
server_key_iv: "163b0fa7a697c2a12f0a273b1bf8ecf3"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:38.345687 27763 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 08:52:38.360886 27778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:52:38.361147 27763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:52:38.361230 27763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "b33042cd01cd458bbcf6c80907f70efa"
format_stamp: "Formatted at 2026-06-19 08:52:38 on dist-test-slave-p0h9"
server_key: "50d13e114ed5b0c3d0d74d8b9513a24c"
server_key_iv: "163b0fa7a697c2a12f0a273b1bf8ecf3"
server_key_version: "kuduclusterkey@0"
I20260619 08:52:38.361320 27763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:52:38.398669 27763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:52:38.401211 27763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:52:38.401414 27763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:52:38.401712 27763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:52:38.402137 27763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:52:38.402232 27763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:52:38.402307 27763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:52:38.402362 27763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:52:38.409377 27763 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:41227
I20260619 08:52:38.410022 27763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:52:38.412541 27891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:41227 every 8 connection(s)
I20260619 08:52:38.418449 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 27763
I20260619 08:52:38.418593 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:52:38.418779 16192 external_mini_cluster.cc:1428] Setting key 7afb143b64ff9ae9fafd67a1bf398866
Jun 19 08:52:38 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859158, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:52:38.424515 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.413389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:38767 (local address 127.15.208.62:37413)
0619 08:52:38.413430 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:38.413432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:38.413443 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:52:38.414147 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:38.414149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:38.414161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:38.414226 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:52:38.414980 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.415714 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.417147 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.417424 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.417467 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.421375 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:52:38.421376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:52:38.421377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:52:38.421384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:52:38.423014 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:52:38.423503 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:52:38.423504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:52:38.423504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:52:38.423559 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:52:38.423813 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:52:38.423814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:52:38.423814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:52:38.423935 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:52:38.423992 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:38.424332 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 08:52:38.424423 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:52:38.425277 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.413100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:37413 (local address 127.15.208.3:38767)
0619 08:52:38.413298 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 08:52:38.413352 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 08:52:38.414013 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:52:38.414305 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:52:38.414309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:52:38.414578 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:52:38.414861 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.414866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.415819 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.415820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.417049 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.417052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.417554 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.417555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.417682 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.419618 (+ 1936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:52:38.419627 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:52:38.421274 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:52:38.423106 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:52:38.423111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:52:38.423113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:52:38.423416 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:52:38.423626 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:52:38.423626 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:52:38.423627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:52:38.423744 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:52:38.424036 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:52:38.424041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:52:38.424192 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 08:52:38.424870 (+ 678us) client_negotiation.cc:240] Negotiation successful
0619 08:52:38.425035 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260619 08:52:38.425875 27892 heartbeater.cc:347] Connected to a master server at 127.15.208.62:37413
I20260619 08:52:38.426014 27892 heartbeater.cc:464] Registering TS with master...
I20260619 08:52:38.426237 27892 heartbeater.cc:511] Master 127.15.208.62:37413 requested a full tablet report, sending...
I20260619 08:52:38.426581 27408 ts_manager.cc:194] Registered new tserver with Master: b33042cd01cd458bbcf6c80907f70efa (127.15.208.3:41227)
I20260619 08:52:38.427063 27408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:38767
I20260619 08:52:38.430202 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:52:38.439733 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.431846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37640 (local address 127.15.208.62:37413)
0619 08:52:38.431883 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:38.431885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:38.432148 (+ 263us) server_negotiation.cc:407] Connection header received
0619 08:52:38.432241 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:38.432242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:38.432251 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:38.432279 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:52:38.433470 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.434168 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.435135 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.435388 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.435425 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.436958 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:52:38.436960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:52:38.436960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:52:38.436967 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:52:38.438578 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:52:38.438950 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:52:38.438951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:52:38.438952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:52:38.439013 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:52:38.439198 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:52:38.439199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:52:38.439199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:52:38.439310 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:52:38.439378 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:38.439571 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 08:52:38.439643 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 08:52:38.441809 27408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:52:38.443992 27408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:52:38.452965 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.447458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.15.208.1:41345)
0619 08:52:38.447667 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:38.447670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:38.448075 (+ 405us) server_negotiation.cc:407] Connection header received
0619 08:52:38.448112 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:38.448114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:38.448129 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:38.448193 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:52:38.448362 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.449481 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.451079 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.452203 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.452270 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.452323 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:38.452827 (+ 504us) server_negotiation.cc:299] Negotiation successful
0619 08:52:38.452889 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":53,"threads_started":1}
I20260619 08:52:38.453583 27904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.446797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:41345 (local address 127.0.0.1:36692)
0619 08:52:38.447957 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0619 08:52:38.447965 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:52:38.448021 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:52:38.448253 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:52:38.448254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:52:38.448256 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:52:38.448298 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.448299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.449575 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.449575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.450970 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.450985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.452493 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.452494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.452569 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.452583 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:52:38.452657 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:52:38.452706 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1147,"mutex_wait_us":7,"thread_start_us":83,"threads_started":1}
I20260619 08:52:38.457168 27553 tablet_service.cc:1467] Processing CreateTablet for tablet 70e8d1a6faad4f83b169b373a8d79733 (DEFAULT_TABLE table=test-table [id=c1c1438d95174182b0a75ed2856b41fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:52:38.457583 27553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70e8d1a6faad4f83b169b373a8d79733. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:52:38.460161 27905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.446999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:41227 (local address 127.0.0.1:35358)
0619 08:52:38.452378 (+ 5379us) negotiation.cc:107] Waiting for socket to connect
0619 08:52:38.452393 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:52:38.452435 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:52:38.454164 (+ 1729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:52:38.454166 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:52:38.454168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:52:38.454203 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.454204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.458015 (+ 3811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.458016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.459071 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.459074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.459965 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.459965 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.460030 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.460035 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:52:38.460062 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 08:52:38.460098 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5362,"thread_start_us":60,"threads_started":1}
I20260619 08:52:38.460773 27907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.447113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35358 (local address 127.15.208.3:41227)
0619 08:52:38.453990 (+ 6877us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:38.453995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:38.454006 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:52:38.454044 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:38.454046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:38.454061 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:38.454267 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:52:38.454339 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.457892 (+ 3553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.459193 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.459860 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.460510 (+ 650us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.460599 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:38.460626 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 08:52:38.460668 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6825,"thread_start_us":84,"threads_started":1}
I20260619 08:52:38.461563 27826 tablet_service.cc:1467] Processing CreateTablet for tablet 70e8d1a6faad4f83b169b373a8d79733 (DEFAULT_TABLE table=test-table [id=c1c1438d95174182b0a75ed2856b41fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:52:38.461865 27826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70e8d1a6faad4f83b169b373a8d79733. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:52:38.462392 27906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.446786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:39707 (local address 127.0.0.1:34608)
0619 08:52:38.447738 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0619 08:52:38.447777 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:52:38.447856 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:52:38.448141 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:52:38.448144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:52:38.448147 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:52:38.448215 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.448216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.449783 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.449784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.451041 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.451044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.462163 (+ 11119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.462164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.462228 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.462232 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:52:38.462285 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:52:38.462325 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":69,"threads_started":1}
I20260619 08:52:38.462689 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.447256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.15.208.2:39707)
0619 08:52:38.447616 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:38.447621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:38.447867 (+ 246us) server_negotiation.cc:407] Connection header received
0619 08:52:38.447944 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:38.447947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:38.447963 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:38.448057 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:52:38.448292 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.449701 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.451163 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.461940 (+ 10777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.462034 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.462124 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:38.462564 (+ 440us) server_negotiation.cc:299] Negotiation successful
0619 08:52:38.462610 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":83,"threads_started":1}
I20260619 08:52:38.463316 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 70e8d1a6faad4f83b169b373a8d79733 (DEFAULT_TABLE table=test-table [id=c1c1438d95174182b0a75ed2856b41fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:52:38.463617 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70e8d1a6faad4f83b169b373a8d79733. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:52:38.466958 27912 tablet_bootstrap.cc:492] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: Bootstrap starting.
I20260619 08:52:38.468080 27912 tablet_bootstrap.cc:654] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: Neither blocks nor log segments found. Creating new log.
I20260619 08:52:38.468386 27912 log.cc:824] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:52:38.468953 27910 tablet_bootstrap.cc:492] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed: Bootstrap starting.
I20260619 08:52:38.469628 27910 tablet_bootstrap.cc:654] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed: Neither blocks nor log segments found. Creating new log.
I20260619 08:52:38.469909 27910 log.cc:824] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed: Log is configured to *not* fsync() on all Append() calls
I20260619 08:52:38.470243 27911 tablet_bootstrap.cc:492] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: Bootstrap starting.
I20260619 08:52:38.471266 27911 tablet_bootstrap.cc:654] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: Neither blocks nor log segments found. Creating new log.
I20260619 08:52:38.471953 27911 log.cc:824] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: Log is configured to *not* fsync() on all Append() calls
I20260619 08:52:38.472712 27912 tablet_bootstrap.cc:492] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: No bootstrap required, opened a new log
I20260619 08:52:38.472805 27912 ts_tablet_manager.cc:1397] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260619 08:52:38.475713 27912 raft_consensus.cc:348] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 } } peers { permanent_uuid: "b33042cd01cd458bbcf6c80907f70efa" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41227 } } peers { permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 } }
I20260619 08:52:38.475895 27912 raft_consensus.cc:374] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:52:38.475932 27912 raft_consensus.cc:729] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62c659a276947ad8fb69afa1acb028e, State: Initialized, Role: FOLLOWER
I20260619 08:52:38.476078 27912 consensus_queue.cc:260] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 } } peers { permanent_uuid: "b33042cd01cd458bbcf6c80907f70efa" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41227 } } peers { permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 } }
I20260619 08:52:38.476320 27912 ts_tablet_manager.cc:1428] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:52:38.476429 27757 heartbeater.cc:503] Master 127.15.208.62:37413 was elected leader, sending a full tablet report...
I20260619 08:52:38.481424 27911 tablet_bootstrap.cc:492] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: No bootstrap required, opened a new log
I20260619 08:52:38.481418 27910 tablet_bootstrap.cc:492] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed: No bootstrap required, opened a new log
I20260619 08:52:38.481506 27910 ts_tablet_manager.cc:1397] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20260619 08:52:38.481504 27911 ts_tablet_manager.cc:1397] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260619 08:52:38.483733 27911 raft_consensus.cc:348] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 } } peers { permanent_uuid: "b33042cd01cd458bbcf6c80907f70efa" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41227 } } peers { permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 } }
I20260619 08:52:38.483934 27911 raft_consensus.cc:374] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:52:38.483990 27911 raft_consensus.cc:729] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b33042cd01cd458bbcf6c80907f70efa, State: Initialized, Role: FOLLOWER
I20260619 08:52:38.484141 27911 consensus_queue.cc:260] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 } } peers { permanent_uuid: "b33042cd01cd458bbcf6c80907f70efa" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41227 } } peers { permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 } }
I20260619 08:52:38.484263 27910 raft_consensus.cc:348] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 } } peers { permanent_uuid: "b33042cd01cd458bbcf6c80907f70efa" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41227 } } peers { permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 } }
I20260619 08:52:38.484418 27911 ts_tablet_manager.cc:1428] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:52:38.484431 27910 raft_consensus.cc:374] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:52:38.484465 27910 raft_consensus.cc:729] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e9dac66343c4706a8d5d2fb901ba6ed, State: Initialized, Role: FOLLOWER
I20260619 08:52:38.484536 27892 heartbeater.cc:503] Master 127.15.208.62:37413 was elected leader, sending a full tablet report...
I20260619 08:52:38.484611 27910 consensus_queue.cc:260] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 } } peers { permanent_uuid: "b33042cd01cd458bbcf6c80907f70efa" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41227 } } peers { permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 } }
I20260619 08:52:38.484885 27910 ts_tablet_manager.cc:1428] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:52:38.485000 27620 heartbeater.cc:503] Master 127.15.208.62:37413 was elected leader, sending a full tablet report...
Jun 19 08:52:38 dist-test-slave-p0h9 krb5kdc[26932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859158, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:52:38.621690 27917 raft_consensus.cc:482] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:52:38.621837 27917 raft_consensus.cc:504] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 } } peers { permanent_uuid: "b33042cd01cd458bbcf6c80907f70efa" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41227 } } peers { permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 } }
I20260619 08:52:38.622318 27917 leader_election.cc:288] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62c659a276947ad8fb69afa1acb028e (127.15.208.2:39707), 5e9dac66343c4706a8d5d2fb901ba6ed (127.15.208.1:41345)
W20260619 08:52:38.623139 27621 tablet.cc:2369] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:52:38.625301 27758 tablet.cc:2369] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:52:38.626085 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.622486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:39707 (local address 127.15.208.3:44013)
0619 08:52:38.622681 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 08:52:38.622693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:52:38.622749 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:52:38.622942 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:52:38.622943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:52:38.622945 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:52:38.623019 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.623021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.623906 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.623907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.625086 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.625089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.625840 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.625841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.625914 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.625918 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:52:38.625960 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 08:52:38.626015 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260619 08:52:38.626554 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.622603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:44013 (local address 127.15.208.2:39707)
0619 08:52:38.622652 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:38.622655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:38.622759 (+ 104us) server_negotiation.cc:407] Connection header received
0619 08:52:38.622810 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:38.622812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:38.622825 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:38.622889 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:52:38.623089 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.623792 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.625197 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.625733 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.625800 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.625809 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:38.626455 (+ 646us) server_negotiation.cc:299] Negotiation successful
0619 08:52:38.626497 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:52:38.626809 27711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e8d1a6faad4f83b169b373a8d79733" candidate_uuid: "b33042cd01cd458bbcf6c80907f70efa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62c659a276947ad8fb69afa1acb028e" is_pre_election: true
I20260619 08:52:38.626938 27711 raft_consensus.cc:2454] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b33042cd01cd458bbcf6c80907f70efa in term 0.
I20260619 08:52:38.627162 27781 leader_election.cc:302] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b33042cd01cd458bbcf6c80907f70efa, c62c659a276947ad8fb69afa1acb028e; no voters:
I20260619 08:52:38.627285 27917 raft_consensus.cc:2781] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:52:38.627327 27917 raft_consensus.cc:482] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:52:38.627351 27917 raft_consensus.cc:3037] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:52:38.628198 27917 raft_consensus.cc:504] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 } } peers { permanent_uuid: "b33042cd01cd458bbcf6c80907f70efa" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41227 } } peers { permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 } }
I20260619 08:52:38.628337 27917 leader_election.cc:288] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [CANDIDATE]: Term 1 election: Requested vote from peers c62c659a276947ad8fb69afa1acb028e (127.15.208.2:39707), 5e9dac66343c4706a8d5d2fb901ba6ed (127.15.208.1:41345)
I20260619 08:52:38.628531 27711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e8d1a6faad4f83b169b373a8d79733" candidate_uuid: "b33042cd01cd458bbcf6c80907f70efa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62c659a276947ad8fb69afa1acb028e"
I20260619 08:52:38.628602 27711 raft_consensus.cc:3037] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:52:38.629642 27711 raft_consensus.cc:2454] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b33042cd01cd458bbcf6c80907f70efa in term 1.
I20260619 08:52:38.629809 27781 leader_election.cc:302] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b33042cd01cd458bbcf6c80907f70efa, c62c659a276947ad8fb69afa1acb028e; no voters:
I20260619 08:52:38.629884 27917 raft_consensus.cc:2781] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:52:38.630033 27917 raft_consensus.cc:686] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 1 LEADER]: Becoming Leader. State: Replica: b33042cd01cd458bbcf6c80907f70efa, State: Running, Role: LEADER
I20260619 08:52:38.630120 27917 consensus_queue.cc:237] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 } } peers { permanent_uuid: "b33042cd01cd458bbcf6c80907f70efa" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41227 } } peers { permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 } }
I20260619 08:52:38.630923 27408 catalog_manager.cc:5495] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa reported cstate change: term changed from 0 to 1, leader changed from <none> to b33042cd01cd458bbcf6c80907f70efa (127.15.208.3). New cstate: current_term: 1 leader_uuid: "b33042cd01cd458bbcf6c80907f70efa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b33042cd01cd458bbcf6c80907f70efa" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 } health_report { overall_health: UNKNOWN } } }
I20260619 08:52:38.635176 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.622535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:41345 (local address 127.15.208.3:40709)
0619 08:52:38.626154 (+ 3619us) negotiation.cc:107] Waiting for socket to connect
0619 08:52:38.626166 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:52:38.626206 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:52:38.631868 (+ 5662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:52:38.631870 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:52:38.631872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:52:38.631911 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.631912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.632764 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.632765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.634161 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:52:38.634164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.634931 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.634932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:52:38.635007 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.635012 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:52:38.635069 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 08:52:38.635109 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3608,"thread_start_us":91,"threads_started":1}
I20260619 08:52:38.635602 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.622904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:40709 (local address 127.15.208.1:41345)
0619 08:52:38.622949 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:38.622951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:38.626329 (+ 3378us) server_negotiation.cc:407] Connection header received
0619 08:52:38.626358 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:38.626359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:38.626369 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:38.626410 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:52:38.632005 (+ 5595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.632640 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.634262 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.634841 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.634921 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.634930 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:38.635487 (+ 557us) server_negotiation.cc:299] Negotiation successful
0619 08:52:38.635539 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:52:38.635895 27558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e8d1a6faad4f83b169b373a8d79733" candidate_uuid: "b33042cd01cd458bbcf6c80907f70efa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" is_pre_election: true
I20260619 08:52:38.636073 27558 raft_consensus.cc:2454] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b33042cd01cd458bbcf6c80907f70efa in term 0.
I20260619 08:52:38.636188 27573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e8d1a6faad4f83b169b373a8d79733" candidate_uuid: "b33042cd01cd458bbcf6c80907f70efa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed"
I20260619 08:52:38.636247 27573 raft_consensus.cc:3037] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:52:38.637383 27573 raft_consensus.cc:2454] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b33042cd01cd458bbcf6c80907f70efa in term 1.
I20260619 08:52:38.643338 27907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.639827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35368 (local address 127.15.208.3:41227)
0619 08:52:38.639869 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:52:38.639871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:52:38.639879 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:52:38.639966 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:52:38.639966 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:52:38.639985 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:52:38.640064 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:52:38.640204 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.640857 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.642074 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:52:38.642374 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:52:38.642443 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:52:38.642682 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:52:38.642822 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:52:38.642908 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:52:38.643217 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 08:52:38.643242 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:52:38.645278 27711 raft_consensus.cc:1261] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e [term 1 FOLLOWER]: Refusing update from remote peer b33042cd01cd458bbcf6c80907f70efa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:52:38.645661 27917 consensus_queue.cc:1035] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62c659a276947ad8fb69afa1acb028e" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 39707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:52:38.645740 27573 raft_consensus.cc:1261] T 70e8d1a6faad4f83b169b373a8d79733 P 5e9dac66343c4706a8d5d2fb901ba6ed [term 1 FOLLOWER]: Refusing update from remote peer b33042cd01cd458bbcf6c80907f70efa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:52:38.646884 27917 consensus_queue.cc:1035] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e9dac66343c4706a8d5d2fb901ba6ed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:52:38.647818 27923 mvcc.cc:205] Tried to move back new op lower bound from 7298495113807884288 to 7298495113749606400. Current Snapshot: MvccSnapshot[applied={T|T < 7298495113807884288}]
I20260619 08:52:38.648613 27924 mvcc.cc:205] Tried to move back new op lower bound from 7298495113807884288 to 7298495113749606400. Current Snapshot: MvccSnapshot[applied={T|T < 7298495113807884288}]
I20260619 08:52:38.652382 27408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:52:38.652467 27408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:52:38.653285 27408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 843928fac50046b6971464ffc6012868: Sending DeleteTablet for 3 replicas of tablet 70e8d1a6faad4f83b169b373a8d79733
I20260619 08:52:38.653591 27553 tablet_service.cc:1514] Processing DeleteTablet for tablet 70e8d1a6faad4f83b169b373a8d79733 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:52:38 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:36692
I20260619 08:52:38.653636 27826 tablet_service.cc:1514] Processing DeleteTablet for tablet 70e8d1a6faad4f83b169b373a8d79733 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:52:38 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:35358
I20260619 08:52:38.653782 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet 70e8d1a6faad4f83b169b373a8d79733 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:52:38 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:34608
I20260619 08:52:38.654388 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 27489
I20260619 08:52:38.661193 27933 tablet_replica.cc:330] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: stopping tablet replica
I20260619 08:52:38.663831 27934 tablet_replica.cc:330] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: stopping tablet replica
I20260619 08:52:38.664026 27934 raft_consensus.cc:2229] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:52:38.664170 27933 raft_consensus.cc:2229] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:52:38.664343 27933 raft_consensus.cc:2258] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:52:38.664486 27934 raft_consensus.cc:2258] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:52:38.665187 27933 ts_tablet_manager.cc:1905] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:52:38.665856 27934 ts_tablet_manager.cc:1905] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:52:38.667953 27933 ts_tablet_manager.cc:1918] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:52:38.668085 27933 log.cc:1196] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/70e8d1a6faad4f83b169b373a8d79733
I20260619 08:52:38.668401 27933 ts_tablet_manager.cc:1939] T 70e8d1a6faad4f83b169b373a8d79733 P b33042cd01cd458bbcf6c80907f70efa: Deleting consensus metadata
I20260619 08:52:38.668972 27394 catalog_manager.cc:4841] TS b33042cd01cd458bbcf6c80907f70efa (127.15.208.3:41227): tablet 70e8d1a6faad4f83b169b373a8d79733 (table test-table [id=c1c1438d95174182b0a75ed2856b41fd]) successfully deleted
I20260619 08:52:38.669278 27934 ts_tablet_manager.cc:1918] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 08:52:38.669327 27395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:41345 (error 108)
I20260619 08:52:38.669349 27934 log.cc:1196] T 70e8d1a6faad4f83b169b373a8d79733 P c62c659a276947ad8fb69afa1acb028e: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/70e8d1a6faad4f83b169b373a8d79733
I20260619 08:52:38.669937 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 27626
I20260619 08:52:38.675354 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 27763
I20260619 08:52:38.675899 27906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.675303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:41345 (local address 127.0.0.1:36698)
0619 08:52:38.675821 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0619 08:52:38.675866 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:41345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492}
W20260619 08:52:38.676793 27395 catalog_manager.cc:4584] TS 5e9dac66343c4706a8d5d2fb901ba6ed (127.15.208.1:41345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70e8d1a6faad4f83b169b373a8d79733: Network error: Client connection negotiation failed: client connection to 127.15.208.1:41345: connect: Connection refused (error 111)
I20260619 08:52:38.677299 27906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:52:38.677171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:39707 (local address 127.0.0.1:34624)
0619 08:52:38.677248 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 08:52:38.677271 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.2:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 08:52:38.681506 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 27376
I20260619 08:52:38.685773 16192 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 08:52:39.030085 16192 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 08:52:39.030195 16192 mini_ranger.cc:66] Stopping Ranger...
I20260619 08:52:39.375065 16192 mini_ranger.cc:68] Stopped Ranger
2026-06-19 08:52:39.375 UTC [26962] LOG: received smart shutdown request
2026-06-19 08:52:39.378 UTC [26962] LOG: background worker "logical replication launcher" (PID 26971) exited with exit code 1
2026-06-19 08:52:39.378 UTC [26966] LOG: shutting down
2026-06-19 08:52:39.528 UTC [26962] LOG: database system is shut down
2026-06-19T08:52:39Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45808 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:52:39 dist-test-slave-p0h9 krb5kdc[27944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:52:39 dist-test-slave-p0h9 krb5kdc[27944](info): set up 2 sockets
Jun 19 08:52:39 dist-test-slave-p0h9 krb5kdc[27944](info): commencing operation
krb5kdc: starting...
W20260619 08:52:40.693223 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:52:40 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:52:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:52:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/rangeradmin_127.15.208.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/rangeradmin_127.15.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/rangerlookup_127.15.208.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/rangerlookup_127.15.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/HTTP_127.15.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/HTTP_127.15.208.20@KRBTEST.COM.keytab.
I20260619 08:52:40.853093 16192 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLqpuKl/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/postgres -l logfile start
2026-06-19 08:52:43.240 UTC [27974] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 08:52:43.240 UTC [27974] LOG: listening on IPv4 address "127.15.208.20", port 54495
2026-06-19 08:52:43.243 UTC [27974] LOG: listening on Unix socket "/tmp/.s.PGSQL.54495"
2026-06-19 08:52:43.278 UTC [27977] LOG: database system was shut down at 2026-06-19 08:52:41 UTC
2026-06-19 08:52:43.282 UTC [27974] LOG: database system is ready to accept connections
I20260619 08:52:44.217079 16192 mini_postgres.cc:96] Postgres bound to 54495
2026-06-19 08:52:44.223 UTC [27985] FATAL: database "slave" does not exist
127.15.208.20:54495 - accepting connections
I20260619 08:52:44.223980 16192 mini_ranger.cc:161] Starting Ranger...
I20260619 08:52:44.238570 16192 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 08:52:44.707865 16192 mini_ranger.cc:87] Created ranger Postgres database
I20260619 08:52:44.707955 16192 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-admin
2026-06-19 08:52:44,984 [I] DB FLAVOR :POSTGRES
2026-06-19 08:52:44,985 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:52:44,985 [I] Checking connection..
2026-06-19 08:52:44,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 08:52:45,243 [I] Checking connection passed.
2026-06-19 08:52:45,244 [I] --------- Verifying version history table ---------
2026-06-19 08:52:45,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:52:45,489 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:52:45,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:52:45,719 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:52:45,719 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 08:52:45,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 08:52:45.959 UTC [28049] WARNING: there is no transaction in progress
2026-06-19 08:52:45,965 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 08:52:45,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:52:46,204 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 08:52:46,205 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:52:46,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 08:52:46,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 08:52:46,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p0h9.c.gcp-upstream.internal','N') ;"
2026-06-19 08:52:46,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:52:47,184 [I] Table x_portal_user does not exist in database ranger
2026-06-19 08:52:47,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:52:47,452 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 08:52:47,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:52:47,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:52:47,943 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 08:52:47,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 08:52:48.869 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:48.884 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:48.900 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:48.915 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:48.932 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:48.952 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49.011 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49.018 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49.031 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49.041 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49.051 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49.065 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49.074 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49.080 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49.086 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49.092 UTC [28157] WARNING: there is no transaction in progress
2026-06-19 08:52:49,685 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 08:52:49,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 08:52:49,931 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 08:52:49,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:52:50,190 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 08:52:50,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:52:50,427 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 08:52:50,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:52:50,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:52:50,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p0h9.c.gcp-upstream.internal';"
2026-06-19 08:52:51,130 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 08:52:51,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 08:52:51,383 [I] DB_PATCHES have already been applied
I20260619 08:52:51.389000 16192 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLqpuKl/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 08:52:51.389065 16192 mini_ranger.cc:193] Using host: 127.15.208.20
I20260619 08:52:51.392161 16192 mini_ranger.cc:227] Ranger admin URL: http://127.15.208.20:50697
Jun 19, 2026 8:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:52:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:52:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:52:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 08:52:52.515784 16192 mini_ranger.cc:160] Time spent starting Ranger: real 8.292s user 0.000s sys 0.009s
Jun 19, 2026 8:53:12 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:53:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 19, 2026 8:53:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:53:12 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:53:13 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 08:53:14.367630 16192 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/rangerkms_127.15.208.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/rangerkms_127.15.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.208.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.208.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.208.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/HTTP_127.15.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.208.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/HTTP_127.15.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 08:53:14 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859194, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 08:53:14.448452 16192 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 08:53:14.457222 16192 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 08:53:15.230396 16192 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 08:53:15.230510 16192 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-kms
I20260619 08:53:15.230546 16192 mini_ranger_kms.cc:227] Using postgres at 127.15.208.20:54495
2026-06-19 08:53:15,398 [I] DB FLAVOR :POSTGRES
2026-06-19 08:53:15,399 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:53:15,399 [I] Checking connection
2026-06-19 08:53:15,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 08:53:15,625 [I] connection success
2026-06-19 08:53:15,625 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 08:53:15,625 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 08:53:15,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 08:53:15,869 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 08:53:15,869 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:53:15,869 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 08:53:15,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.208.20:54495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 08:53:16,116 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 08:53:16.617882 16192 mini_ranger_kms.cc:310] Created kms service
I20260619 08:53:16.781457 16192 mini_ranger_kms.cc:326] Created kudu user
I20260619 08:53:16.822564 16192 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 08:53:16 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859194, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:53:17.224316 16192 mini_ranger_kms.cc:384] Added ranger policy
I20260619 08:53:17.224896 16192 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLqpuKl/build/release/bin/postgresql.jar:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLqpuKl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 08:53:17.224946 16192 mini_ranger_kms.cc:240] Using host: 127.15.208.20
I20260619 08:53:17.229176 16192 mini_ranger_kms.cc:276] Ranger KMS PID: 28337
I20260619 08:53:17.229249 16192 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.208.20:60859
Jun 19, 2026 8:53:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:53:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:53:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:53:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:53:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:53:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 08:53:19.451525 16192 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.003s user 0.000s sys 0.150s
I20260619 08:53:19.451642 16192 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 08:53:19.451699 16192 mini_ranger_kms.cc:401] 127.15.208.20:60859/kms/v1/keys
Jun 19 08:53:19 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781859199, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.208.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 08:53:19 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859199, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.208.20@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
Jun 19 08:53:20 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859199, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.208.20@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
Jun 19 08:53:22 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:53:22.739416 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46857
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:46857
--ranger_config_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60859/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:22.823287 28388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:22.823508 28388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:22.823531 28388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:22.824816 28388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:53:22.824859 28388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:22.824873 28388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:22.824908 28388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:53:22.824940 28388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:53:22.826459 28388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.208.20:60859/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:46857
--ranger_config_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:22.826704 28388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:22.826925 28388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:22.832086 28394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:22.832149 28393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:22.832122 28388 server_base.cc:1034] running on GCE node
W20260619 08:53:22.832103 28396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:22.832463 28388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:22.832708 28388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:22.833901 28388 hybrid_clock.cc:648] HybridClock initialized: now 1781859202833882 us; error 37 us; skew 500 ppm
Jun 19 08:53:22 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859202, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:22.836403 28388 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:53:22.836944 28388 webserver.cc:460] Webserver started at http://127.15.208.62:35121/ using document root <none> and password file <none>
I20260619 08:53:22.837177 28388 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:22.837218 28388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:22.837366 28388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:53:22 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859202, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:53:22.897691 28388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "31b094197162448597e78b3dde583f24"
format_stamp: "Formatted at 2026-06-19 08:53:22 on dist-test-slave-p0h9"
server_key: "9de607a0469c7d4a28ed52f15779b5d8"
server_key_iv: "f3678f67c7134d9465f2bfd0ef2d0479"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:22.898103 28388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "31b094197162448597e78b3dde583f24"
format_stamp: "Formatted at 2026-06-19 08:53:22 on dist-test-slave-p0h9"
server_key: "9de607a0469c7d4a28ed52f15779b5d8"
server_key_iv: "f3678f67c7134d9465f2bfd0ef2d0479"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:22.899497 28388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:53:22.914201 28403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:22.914391 28388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:22.914485 28388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "31b094197162448597e78b3dde583f24"
format_stamp: "Formatted at 2026-06-19 08:53:22 on dist-test-slave-p0h9"
server_key: "9de607a0469c7d4a28ed52f15779b5d8"
server_key_iv: "f3678f67c7134d9465f2bfd0ef2d0479"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:22.914588 28388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:22.937065 28388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:22.939240 28388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:22.939406 28388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:22.944563 28388 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:46857
I20260619 08:53:22.944859 28455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:46857 every 8 connection(s)
I20260619 08:53:22.945067 28388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:53:22.945750 28456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:22.945945 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 28388
I20260619 08:53:22.946053 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:53:22.946240 16192 external_mini_cluster.cc:1428] Setting key b7cc2d8a6cb6576002c778db7d539ff2
I20260619 08:53:22.949244 28456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: Bootstrap starting.
I20260619 08:53:22.949883 28456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:22.950210 28456 log.cc:824] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:22.951047 28456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: No bootstrap required, opened a new log
Jun 19 08:53:22 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:22.953549 28456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } }
I20260619 08:53:22.953714 28456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:22.953745 28456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31b094197162448597e78b3dde583f24, State: Initialized, Role: FOLLOWER
I20260619 08:53:22.953864 28456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } }
I20260619 08:53:22.953922 28456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:53:22.953950 28456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:53:22.953985 28456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:22.954627 28456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } }
I20260619 08:53:22.954721 28456 leader_election.cc:302] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31b094197162448597e78b3dde583f24; no voters:
I20260619 08:53:22.954880 28456 leader_election.cc:288] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:53:22.955117 28456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:53:22.955165 28461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:53:22.955313 28461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 1 LEADER]: Becoming Leader. State: Replica: 31b094197162448597e78b3dde583f24, State: Running, Role: LEADER
I20260619 08:53:22.955397 28461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } }
I20260619 08:53:22.955739 28461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31b094197162448597e78b3dde583f24. Latest consensus state: current_term: 1 leader_uuid: "31b094197162448597e78b3dde583f24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } } }
I20260619 08:53:22.955818 28461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:22.956015 28470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:53:22.956522 28470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:53:22.956976 28456 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 08:53:22.957652 28462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31b094197162448597e78b3dde583f24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } } }
I20260619 08:53:22.957795 28462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:22.958323 28459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:22.946746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55280 (local address 127.15.208.62:46857)
0619 08:53:22.946981 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:22.946990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:22.947020 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:53:22.947598 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:22.947607 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:22.947668 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:22.947829 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:22.948718 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:22.950108 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:22.951158 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:22.951573 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:22.951699 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:22.954173 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:22.954177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:22.954177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:22.954193 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:22.956101 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:22.956550 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:22.956553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:22.956553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:22.956647 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:22.956940 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:22.956941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:22.956941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:22.957256 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:22.957914 (+ 658us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:22.958008 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 08:53:22.958125 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":100,"threads_started":1}
I20260619 08:53:22.962591 28470 catalog_manager.cc:1347] Generated new cluster ID: 5b0262f1e7e64ee391bcf32b96bc6265
I20260619 08:53:22.962699 28470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:53:22.971240 28470 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:53:22.972043 28470 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:53:22.981961 28470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: Generated new TSK 0
I20260619 08:53:22.982158 28470 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:53:23.691573 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46857
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:23.780468 28501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:23.780830 28501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:23.780932 28501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:23.782148 28501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:23.782215 28501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:23.782262 28501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:53:23.783762 28501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.208.20:60859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:23.784025 28501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:23.784246 28501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:23.790167 28507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:23.790151 28506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:23.790457 28501 server_base.cc:1034] running on GCE node
W20260619 08:53:23.790258 28509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:23.790772 28501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:23.791085 28501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:23.792253 28501 hybrid_clock.cc:648] HybridClock initialized: now 1781859203792243 us; error 30 us; skew 500 ppm
Jun 19 08:53:23 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859203, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:23.795571 28501 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:53:23.796111 28501 webserver.cc:460] Webserver started at http://127.15.208.1:40439/ using document root <none> and password file <none>
I20260619 08:53:23.796420 28501 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:23.796487 28501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:23.796662 28501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:53:23 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859203, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:53:23.811031 28501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "f54ff5e747f44f479351058aed0559f3"
format_stamp: "Formatted at 2026-06-19 08:53:23 on dist-test-slave-p0h9"
server_key: "29dbd3e5cc6431503d8e0547449d3eaf"
server_key_iv: "c60f1949872b48c3cb09d2242b9e7fbb"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:23.811477 28501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "f54ff5e747f44f479351058aed0559f3"
format_stamp: "Formatted at 2026-06-19 08:53:23 on dist-test-slave-p0h9"
server_key: "29dbd3e5cc6431503d8e0547449d3eaf"
server_key_iv: "c60f1949872b48c3cb09d2242b9e7fbb"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:23.817232 28501 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260619 08:53:23.834399 28516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:23.834652 28501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:23.834753 28501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "f54ff5e747f44f479351058aed0559f3"
format_stamp: "Formatted at 2026-06-19 08:53:23 on dist-test-slave-p0h9"
server_key: "29dbd3e5cc6431503d8e0547449d3eaf"
server_key_iv: "c60f1949872b48c3cb09d2242b9e7fbb"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:23.834842 28501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:23.855476 28501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:23.858276 28501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:23.858435 28501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:23.858671 28501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:23.859076 28501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:23.859124 28501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:23.859155 28501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:23.859206 28501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:23.869309 28501 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:39177
I20260619 08:53:23.869611 28630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:39177 every 8 connection(s)
I20260619 08:53:23.869767 28501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:53:23.874744 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 28501
I20260619 08:53:23.874848 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:53:23.875020 16192 external_mini_cluster.cc:1428] Setting key 03f1f9cfe64e1b7a17a42f6d6eb71485
Jun 19 08:53:23 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859203, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:23.889830 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:23.870713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46857 (local address 127.15.208.1:42659)
0619 08:53:23.871571 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:23.872892 (+ 1321us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:23.874077 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:23.874479 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:23.874484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:23.878586 (+ 4102us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:23.878886 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:23.878894 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:23.879924 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:23.879925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:23.881229 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:23.881232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:23.883135 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:23.883136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:23.883295 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:23.883909 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:23.883922 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:23.885712 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:23.887908 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:23.887915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:23.887918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:23.888279 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:23.888516 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:23.888517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:23.888517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:23.888660 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:23.889038 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:23.889045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:23.889247 (+ 202us) client_negotiation.cc:769] Sending connection context
0619 08:53:23.889358 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 08:53:23.889567 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"thread_start_us":83,"threads_started":1}
I20260619 08:53:23.890461 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:23.870754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:42659 (local address 127.15.208.62:46857)
0619 08:53:23.871884 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:23.871887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:23.873377 (+ 1490us) server_negotiation.cc:407] Connection header received
0619 08:53:23.874243 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:23.874245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:23.874258 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:23.874321 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:23.879033 (+ 4712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:23.879810 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:23.881343 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:23.882943 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:23.883043 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:23.885824 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:23.885825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:23.885826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:23.885833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:23.887798 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:23.888370 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:23.888371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:23.888372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:23.888440 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:23.888773 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:23.888774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:23.888774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:23.888927 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:23.889008 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:23.890270 (+ 1262us) server_negotiation.cc:299] Negotiation successful
0619 08:53:23.890359 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1099,"thread_start_us":84,"threads_started":1}
I20260619 08:53:23.891350 28631 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46857
I20260619 08:53:23.891455 28631 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:23.891654 28631 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
I20260619 08:53:23.892302 28420 ts_manager.cc:194] Registered new tserver with Master: f54ff5e747f44f479351058aed0559f3 (127.15.208.1:39177)
I20260619 08:53:23.893273 28420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:42659
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:53:23.929333 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46857
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:24.048866 28638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:24.049232 28638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:24.049325 28638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:24.051020 28638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:24.051141 28638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:24.051239 28638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:53:24.053539 28638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.208.20:60859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:24.054042 28638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:24.054406 28638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:24.061380 28646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:24.061416 28648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:24.062363 28645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:24.062621 28638 server_base.cc:1034] running on GCE node
I20260619 08:53:24.063024 28638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:24.063308 28638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:24.064477 28638 hybrid_clock.cc:648] HybridClock initialized: now 1781859204064456 us; error 35 us; skew 500 ppm
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:24.067858 28638 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:53:24.068401 28638 webserver.cc:460] Webserver started at http://127.15.208.2:38891/ using document root <none> and password file <none>
I20260619 08:53:24.068665 28638 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:24.068720 28638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:24.068877 28638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:53:24.092689 28638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "9278c6f21d064a7987b7def0889079f6"
format_stamp: "Formatted at 2026-06-19 08:53:24 on dist-test-slave-p0h9"
server_key: "51711fa62447dd12a8eed0f1ee793159"
server_key_iv: "6a4be9bd37ae83be9f95d1d3f0821a05"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:24.094424 28638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "9278c6f21d064a7987b7def0889079f6"
format_stamp: "Formatted at 2026-06-19 08:53:24 on dist-test-slave-p0h9"
server_key: "51711fa62447dd12a8eed0f1ee793159"
server_key_iv: "6a4be9bd37ae83be9f95d1d3f0821a05"
server_key_version: "kuduclusterkey@0"
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:24.097488 28638 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 08:53:24.118727 28655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.119719 28638 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260619 08:53:24.119824 28638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "9278c6f21d064a7987b7def0889079f6"
format_stamp: "Formatted at 2026-06-19 08:53:24 on dist-test-slave-p0h9"
server_key: "51711fa62447dd12a8eed0f1ee793159"
server_key_iv: "6a4be9bd37ae83be9f95d1d3f0821a05"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:24.119940 28638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:24.142692 28638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:24.145290 28638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:24.145660 28638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:24.145942 28638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:24.146342 28638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:24.146432 28638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.146503 28638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:24.146560 28638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.153239 28638 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:44759
I20260619 08:53:24.153743 28638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:53:24.156020 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 28638
I20260619 08:53:24.156116 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:53:24.156322 16192 external_mini_cluster.cc:1428] Setting key 7b5b358c0e6df73882c4fadbc4531b73
I20260619 08:53:24.156360 28768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:44759 every 8 connection(s)
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:24.175045 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:24.157102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:49503 (local address 127.15.208.62:46857)
0619 08:53:24.157168 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:24.157170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:24.157385 (+ 215us) server_negotiation.cc:407] Connection header received
0619 08:53:24.161242 (+ 3857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:24.161243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:24.161261 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:24.161293 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:24.162161 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.162939 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.166271 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.166595 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.166635 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:24.170889 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:24.170891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:24.170891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:24.170900 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:24.172615 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:24.173937 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:24.173938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:24.173938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:24.174007 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:24.174264 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:24.174265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:24.174265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:24.174410 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:24.174461 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:24.174800 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 08:53:24.174903 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:53:24.175720 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:24.157032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46857 (local address 127.15.208.2:49503)
0619 08:53:24.157246 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:24.157297 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:24.161090 (+ 3793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:24.161493 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:24.161497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:24.161751 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:24.162041 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:24.162046 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.164876 (+ 2830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.164877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:24.166161 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:24.166164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.166733 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.166733 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:24.166915 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:24.167606 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:24.167616 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:24.170778 (+ 3162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:24.173541 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:24.173548 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:24.173550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:24.173851 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:24.174072 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:24.174073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:24.174073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:24.174192 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:24.174492 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:24.174497 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:24.174659 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 08:53:24.175312 (+ 653us) client_negotiation.cc:240] Negotiation successful
0619 08:53:24.175484 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":88,"threads_started":1}
I20260619 08:53:24.176295 28769 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46857
I20260619 08:53:24.176409 28769 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:24.176606 28769 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
I20260619 08:53:24.176988 28420 ts_manager.cc:194] Registered new tserver with Master: 9278c6f21d064a7987b7def0889079f6 (127.15.208.2:44759)
I20260619 08:53:24.177470 28420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:49503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:53:24.187631 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46857
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:24.289266 28774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:24.289520 28774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:24.289563 28774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:24.290829 28774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:24.290891 28774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:24.290944 28774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:53:24.292466 28774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.208.20:60859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:24.292730 28774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:24.293022 28774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:24.299044 28781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:24.299479 28783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:24.299597 28780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:24.300096 28774 server_base.cc:1034] running on GCE node
I20260619 08:53:24.300330 28774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:24.300647 28774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:24.301823 28774 hybrid_clock.cc:648] HybridClock initialized: now 1781859204301794 us; error 45 us; skew 500 ppm
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:24.305282 28774 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:53:24.305863 28774 webserver.cc:460] Webserver started at http://127.15.208.3:33875/ using document root <none> and password file <none>
I20260619 08:53:24.306223 28774 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:24.306334 28774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:24.306536 28774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:53:24.320330 28774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "c43683b727c64ce9affe413c7ea39970"
format_stamp: "Formatted at 2026-06-19 08:53:24 on dist-test-slave-p0h9"
server_key: "8fd0acb2097b317a029b8a0e7091bb89"
server_key_iv: "272735ba4d8d80252860bac210404e9a"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:24.320948 28774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "c43683b727c64ce9affe413c7ea39970"
format_stamp: "Formatted at 2026-06-19 08:53:24 on dist-test-slave-p0h9"
server_key: "8fd0acb2097b317a029b8a0e7091bb89"
server_key_iv: "272735ba4d8d80252860bac210404e9a"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:24.325203 28774 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.001s
I20260619 08:53:24.339205 28790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.339484 28774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:53:24.339696 28774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "c43683b727c64ce9affe413c7ea39970"
format_stamp: "Formatted at 2026-06-19 08:53:24 on dist-test-slave-p0h9"
server_key: "8fd0acb2097b317a029b8a0e7091bb89"
server_key_iv: "272735ba4d8d80252860bac210404e9a"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:24.339905 28774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:24.352653 28774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:24.354852 28774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:24.355013 28774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:24.355351 28774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:24.355784 28774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:24.355829 28774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.355890 28774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:24.355918 28774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.362481 28774 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:44323
I20260619 08:53:24.362937 28774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:53:24.370345 28903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:44323 every 8 connection(s)
I20260619 08:53:24.372466 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 28774
I20260619 08:53:24.372591 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:53:24.372833 16192 external_mini_cluster.cc:1428] Setting key a5fa869823511b5028b1a0245abb91a3
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:24.384344 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:24.372188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:34709 (local address 127.15.208.62:46857)
0619 08:53:24.372266 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:24.372268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:24.374157 (+ 1889us) server_negotiation.cc:407] Connection header received
0619 08:53:24.374761 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:24.374762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:24.374776 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:24.374853 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:24.375401 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.376194 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.377612 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.377911 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.377958 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:24.380851 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:24.380853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:24.380853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:24.380862 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:24.382631 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:24.383169 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:24.383170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:24.383171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:24.383253 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:24.383584 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:24.383585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:24.383586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:24.383728 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:24.383792 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:24.384157 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 08:53:24.384244 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:53:24.385154 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:24.372144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46857 (local address 127.15.208.3:34709)
0619 08:53:24.374005 (+ 1861us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:24.374060 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:24.374650 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:24.374912 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:24.374915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:24.375106 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:24.375310 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:24.375314 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.376296 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.376297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:24.377501 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:24.377505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.378063 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.378064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:24.378207 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:24.378820 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:24.378830 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:24.380761 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:24.382723 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:24.382730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:24.382733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:24.383056 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:24.383337 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:24.383337 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:24.383338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:24.383500 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:24.383823 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:24.383830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:24.383997 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 08:53:24.384690 (+ 693us) client_negotiation.cc:240] Negotiation successful
0619 08:53:24.384914 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1805,"thread_start_us":85,"threads_started":1}
I20260619 08:53:24.385664 28904 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46857
I20260619 08:53:24.385766 28904 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:24.385965 28904 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
I20260619 08:53:24.386371 28420 ts_manager.cc:194] Registered new tserver with Master: c43683b727c64ce9affe413c7ea39970 (127.15.208.3:44323)
I20260619 08:53:24.386885 28420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:34709
I20260619 08:53:24.395347 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:53:24.395927 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 28501
I20260619 08:53:24.400869 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 28638
I20260619 08:53:24.405061 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 28774
I20260619 08:53:24.410336 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 28388
I20260619 08:53:24.416177 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46857
--webserver_interface=127.15.208.62
--webserver_port=35121
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:46857
--ranger_config_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60859/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.208.20:60859/kms with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:24.496771 28907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:24.496968 28907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:24.496989 28907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:24.498149 28907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:53:24.498190 28907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:24.498205 28907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:24.498255 28907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:53:24.498273 28907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:53:24.499845 28907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.208.20:60859/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:46857
--ranger_config_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=35121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:24.500079 28907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:24.500303 28907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:24.504987 28913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:24.505003 28915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:24.505024 28912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:24.505230 28907 server_base.cc:1034] running on GCE node
I20260619 08:53:24.505475 28907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:24.505776 28907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:24.506942 28907 hybrid_clock.cc:648] HybridClock initialized: now 1781859204506921 us; error 32 us; skew 500 ppm
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:24.509351 28907 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:53:24.509792 28907 webserver.cc:460] Webserver started at http://127.15.208.62:35121/ using document root <none> and password file <none>
I20260619 08:53:24.510005 28907 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:24.510053 28907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:53:24.581813 28907 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:53:24.582759 28923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.582975 28907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:53:24.583076 28907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "31b094197162448597e78b3dde583f24"
format_stamp: "Formatted at 2026-06-19 08:53:22 on dist-test-slave-p0h9"
server_key: "9de607a0469c7d4a28ed52f15779b5d8"
server_key_iv: "f3678f67c7134d9465f2bfd0ef2d0479"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:24.583557 28907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:24.598731 28907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:24.600294 28907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:24.600456 28907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:24.604754 28907 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:46857
I20260619 08:53:24.604805 28975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:46857 every 8 connection(s)
I20260619 08:53:24.605191 28907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:53:24.606377 28976 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:53:24.606942 28976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: Bootstrap starting.
I20260619 08:53:24.611990 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 28907
I20260619 08:53:24.612280 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:39177
--local_ip_for_outbound_sockets=127.15.208.1
--tserver_master_addrs=127.15.208.62:46857
--webserver_port=40439
--webserver_interface=127.15.208.1
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.208.20:60859/kms with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:53:24.615051 28976 log.cc:824] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:24.616807 28976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:53:24.617091 28976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: Bootstrap complete.
I20260619 08:53:24.619465 28976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } }
I20260619 08:53:24.619690 28976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31b094197162448597e78b3dde583f24, State: Initialized, Role: FOLLOWER
I20260619 08:53:24.619849 28976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } }
I20260619 08:53:24.619923 28976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:53:24.619968 28976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:53:24.620025 28976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:53:24.621057 28976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } }
I20260619 08:53:24.621148 28976 leader_election.cc:302] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31b094197162448597e78b3dde583f24; no voters:
I20260619 08:53:24.621332 28976 leader_election.cc:288] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:53:24.621366 28980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:53:24.621474 28980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [term 2 LEADER]: Becoming Leader. State: Replica: 31b094197162448597e78b3dde583f24, State: Running, Role: LEADER
I20260619 08:53:24.621598 28976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:53:24.621650 28980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } }
I20260619 08:53:24.621999 28981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "31b094197162448597e78b3dde583f24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } } }
I20260619 08:53:24.622014 28982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31b094197162448597e78b3dde583f24. Latest consensus state: current_term: 2 leader_uuid: "31b094197162448597e78b3dde583f24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b094197162448597e78b3dde583f24" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46857 } } }
I20260619 08:53:24.622179 28981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:24.622186 28982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:24.622428 28989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:53:24.622998 28989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:53:24.623199 28989 catalog_manager.cc:1259] Loaded cluster ID: 5b0262f1e7e64ee391bcf32b96bc6265
I20260619 08:53:24.623278 28989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:53:24.623342 28976 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 08:53:24.624495 28989 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:53:24.624668 28989 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: Loaded TSK: 0
I20260619 08:53:24.624897 28989 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:53:24.724169 28978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:24.724452 28978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:24.724488 28978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:24.726248 28978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:24.726357 28978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:24.726440 28978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:53:24.728686 28978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.208.20:60859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:39177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=40439
--tserver_master_addrs=127.15.208.62:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:24.728965 28978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:24.729264 28978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:24.736431 29013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:24.736444 29016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:24.736454 29014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:24.737769 28978 server_base.cc:1034] running on GCE node
I20260619 08:53:24.737982 28978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:24.738237 28978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:24.739408 28978 hybrid_clock.cc:648] HybridClock initialized: now 1781859204739373 us; error 51 us; skew 500 ppm
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:24.742654 28978 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:53:24.743198 28978 webserver.cc:460] Webserver started at http://127.15.208.1:40439/ using document root <none> and password file <none>
I20260619 08:53:24.743453 28978 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:24.743546 28978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:53:24.769183 28978 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.001s sys 0.000s
I20260619 08:53:24.770500 29024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.770697 28978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:24.770783 28978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "f54ff5e747f44f479351058aed0559f3"
format_stamp: "Formatted at 2026-06-19 08:53:23 on dist-test-slave-p0h9"
server_key: "29dbd3e5cc6431503d8e0547449d3eaf"
server_key_iv: "c60f1949872b48c3cb09d2242b9e7fbb"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:24.771848 28978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:24.803715 28978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:24.805955 28978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:24.806121 28978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:24.806375 28978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:24.806748 28978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:24.806799 28978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.806834 28978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:24.806861 28978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.814364 28978 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:39177
I20260619 08:53:24.814867 28978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:53:24.815412 29137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:39177 every 8 connection(s)
I20260619 08:53:24.818837 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 28978
I20260619 08:53:24.819131 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:44759
--local_ip_for_outbound_sockets=127.15.208.2
--tserver_master_addrs=127.15.208.62:46857
--webserver_port=38891
--webserver_interface=127.15.208.2
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.208.20:60859/kms with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:24.829903 29140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:24.815674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46857 (local address 127.15.208.1:51343)
0619 08:53:24.815881 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:24.815942 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:24.817285 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:24.817999 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:24.818004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:24.818235 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:24.818475 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:24.818480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.820165 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.820167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:24.821596 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:24.821600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.822588 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.822590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:24.822738 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:24.823529 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:24.823541 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:24.825163 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:24.827700 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:24.827707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:24.827710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:24.828041 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:24.828343 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:24.828344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:24.828344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:24.828501 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:24.829098 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:24.829105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:24.829296 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 08:53:24.829408 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:53:24.829605 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260619 08:53:24.830816 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:24.816069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:51343 (local address 127.15.208.62:46857)
0619 08:53:24.816300 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:24.816310 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:24.816345 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:53:24.817614 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:24.817621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:24.817681 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:24.817812 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:24.818616 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.820042 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.821697 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:24.822446 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:24.822612 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:24.825302 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:24.825308 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:24.825308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:24.825321 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:24.827596 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:24.828141 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:24.828145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:24.828145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:24.828262 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:24.828587 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:24.828588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:24.828588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:24.828969 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:24.829064 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:24.830396 (+ 1332us) server_negotiation.cc:299] Negotiation successful
0619 08:53:24.830559 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":83,"threads_started":1}
I20260619 08:53:24.832120 29138 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46857
I20260619 08:53:24.832219 29138 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:24.832425 29138 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
W20260619 08:53:24.833442 29138 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:46857 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 08:53:24.833519 29138 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:24.833576 29138 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
I20260619 08:53:24.834010 29138 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:24.834086 29138 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
W20260619 08:53:24.834421 29138 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:53:24.927107 29142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:24.927454 29142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:24.927543 29142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:24.928719 29142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:24.928828 29142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:24.928872 29142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:53:24.930393 29142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.208.20:60859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:44759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=38891
--tserver_master_addrs=127.15.208.62:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:24.930661 29142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:24.930883 29142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:24.935985 29150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:24.935985 29147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:24.936048 29148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:24.937024 29142 server_base.cc:1034] running on GCE node
I20260619 08:53:24.937238 29142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:24.937502 29142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:24.938676 29142 hybrid_clock.cc:648] HybridClock initialized: now 1781859204938662 us; error 35 us; skew 500 ppm
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:24.941380 29142 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:53:24.941938 29142 webserver.cc:460] Webserver started at http://127.15.208.2:38891/ using document root <none> and password file <none>
I20260619 08:53:24.942250 29142 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:24.942384 29142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:53:24 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:53:24.965095 29142 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.001s
I20260619 08:53:24.966012 29158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:24.966208 29142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:53:24.966313 29142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "9278c6f21d064a7987b7def0889079f6"
format_stamp: "Formatted at 2026-06-19 08:53:24 on dist-test-slave-p0h9"
server_key: "51711fa62447dd12a8eed0f1ee793159"
server_key_iv: "6a4be9bd37ae83be9f95d1d3f0821a05"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:24.966717 29142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:25.003419 29142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:25.006079 29142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:25.006278 29142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:25.006572 29142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:25.006955 29142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:25.007041 29142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:25.007104 29142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:25.007169 29142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:25.014855 29142 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:44759
I20260619 08:53:25.014961 29271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:44759 every 8 connection(s)
I20260619 08:53:25.015313 29142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:53:25 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:25.024540 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 29142
I20260619 08:53:25.024907 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.208.20:60859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:44323
--local_ip_for_outbound_sockets=127.15.208.3
--tserver_master_addrs=127.15.208.62:46857
--webserver_port=33875
--webserver_interface=127.15.208.3
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.208.20:60859/kms with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:53:25.026292 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:25.016391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:33687 (local address 127.15.208.62:46857)
0619 08:53:25.016530 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:25.016532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:25.016543 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:53:25.017134 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:25.017135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:25.017147 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:25.017201 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:25.017891 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:25.018379 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:25.019861 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:25.020049 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:25.020098 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:25.022953 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:25.022955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:25.022955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:25.022963 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:25.024580 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:25.025120 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:25.025121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:25.025122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:25.025187 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:25.025477 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:25.025478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:25.025478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:25.025650 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:25.025710 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:25.026105 (+ 395us) server_negotiation.cc:299] Negotiation successful
0619 08:53:25.026195 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260619 08:53:25.026821 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:25.016099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46857 (local address 127.15.208.2:33687)
0619 08:53:25.016290 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:25.016341 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:25.017020 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:25.017281 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:25.017286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:25.017524 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:25.017802 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:25.017809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:25.018478 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:25.018480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:25.019775 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:25.019778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:25.020152 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:25.020154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:25.020319 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:25.021041 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:25.021050 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:25.022774 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:25.024677 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:25.024684 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:25.024686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:25.025018 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:25.025260 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:25.025261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:25.025261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:25.025400 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:25.025753 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:25.025759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:25.025920 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 08:53:25.026422 (+ 502us) client_negotiation.cc:240] Negotiation successful
0619 08:53:25.026590 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260619 08:53:25.027758 29272 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46857
I20260619 08:53:25.027864 29272 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:25.028055 29272 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
W20260619 08:53:25.032913 29272 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:46857 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 08:53:25.033012 29272 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:25.033149 29272 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
I20260619 08:53:25.033619 29272 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:25.033715 29272 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
W20260619 08:53:25.034111 29272 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:53:25.124042 29275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:25.124277 29275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:25.124320 29275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:25.125658 29275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:25.125720 29275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:25.125767 29275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:53:25.127235 29275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.208.20:60859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:44323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=33875
--tserver_master_addrs=127.15.208.62:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:25.127552 29275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:25.127837 29275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:25.134593 29281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:25.134783 29284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:25.134593 29282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:25.135988 29275 server_base.cc:1034] running on GCE node
I20260619 08:53:25.136353 29275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:25.136626 29275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:25.137789 29275 hybrid_clock.cc:648] HybridClock initialized: now 1781859205137754 us; error 48 us; skew 500 ppm
Jun 19 08:53:25 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:25.141360 29275 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:53:25.141865 29275 webserver.cc:460] Webserver started at http://127.15.208.3:33875/ using document root <none> and password file <none>
I20260619 08:53:25.142102 29275 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:25.142185 29275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:53:25 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:53:25.157136 29275 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 08:53:25.157939 29292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:25.158134 29275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:53:25.158270 29275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "c43683b727c64ce9affe413c7ea39970"
format_stamp: "Formatted at 2026-06-19 08:53:24 on dist-test-slave-p0h9"
server_key: "8fd0acb2097b317a029b8a0e7091bb89"
server_key_iv: "272735ba4d8d80252860bac210404e9a"
server_key_version: "kuduclusterkey@0"
I20260619 08:53:25.158648 29275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:25.175199 29275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:25.177280 29275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:25.177448 29275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:25.177758 29275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:25.178100 29275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:25.178154 29275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:25.178202 29275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:25.178229 29275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:25.185549 29275 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:44323
I20260619 08:53:25.185624 29405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:44323 every 8 connection(s)
I20260619 08:53:25.185961 29275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:53:25 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:25.192932 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 29275
I20260619 08:53:25.196417 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:25.186737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:46827 (local address 127.15.208.62:46857)
0619 08:53:25.186814 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:25.186817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:25.187017 (+ 200us) server_negotiation.cc:407] Connection header received
0619 08:53:25.187566 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:25.187568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:25.187581 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:25.187637 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:25.188230 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:25.189092 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:25.190278 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:25.190583 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:25.190668 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:25.193142 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:25.193144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:25.193145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:25.193153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:25.194787 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:25.195280 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:25.195281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:25.195282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:25.195338 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:25.195658 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:25.195658 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:25.195659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:25.195802 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:25.195870 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:25.196236 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 08:53:25.196324 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:53:25.196623 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:25.186726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46857 (local address 127.15.208.3:46827)
0619 08:53:25.186891 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:25.186934 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:25.187442 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:25.187716 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:25.187720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:25.187906 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:25.188124 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:25.188130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:25.189188 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:25.189190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:25.190174 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:25.190177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:25.190688 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:25.190689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:25.190785 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:25.191349 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:25.191357 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:25.193012 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:25.194880 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:25.194888 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:25.194890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:25.195197 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:25.195401 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:25.195402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:25.195402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:25.195579 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:25.195901 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:25.195908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:25.196089 (+ 181us) client_negotiation.cc:769] Sending connection context
0619 08:53:25.196213 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 08:53:25.196365 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260619 08:53:25.197221 29406 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46857
I20260619 08:53:25.197377 29406 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:25.197593 29406 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
W20260619 08:53:25.198231 29406 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:46857 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 08:53:25.198309 29406 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:25.198382 29406 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
I20260619 08:53:25.198675 29406 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:25.198750 29406 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
W20260619 08:53:25.199018 29406 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:53:25.205019 29410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:25.193394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.15.208.62:46857)
0619 08:53:25.193743 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:25.193745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:25.193752 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:53:25.193775 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:25.193776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:25.193788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:25.193858 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:25.194655 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:25.195430 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:25.196965 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:25.197273 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:25.197338 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:25.202225 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:25.202226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:25.202227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:25.202234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:25.203683 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:25.204105 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:25.204106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:25.204107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:25.204166 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:25.204405 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:25.204406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:25.204406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:25.204530 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:25.204716 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:25.204774 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 08:53:25.204869 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":88,"threads_started":1}
Jun 19 08:53:25 dist-test-slave-p0h9 krb5kdc[27944](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781859205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:25.834625 29138 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:25.834831 29138 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
I20260619 08:53:25.835333 28939 ts_manager.cc:194] Registered new tserver with Master: f54ff5e747f44f479351058aed0559f3 (127.15.208.1:39177)
I20260619 08:53:25.836206 28939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:51343
I20260619 08:53:26.034287 29272 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:26.034471 29272 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
I20260619 08:53:26.034750 28939 ts_manager.cc:194] Registered new tserver with Master: 9278c6f21d064a7987b7def0889079f6 (127.15.208.2:44759)
I20260619 08:53:26.035285 28939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:33687
Jun 19 08:53:26 dist-test-slave-p0h9 krb5kdc[27944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for HTTP/127.15.208.20@KRBTEST.COM
I20260619 08:53:26.199206 29406 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:26.199363 29406 heartbeater.cc:511] Master 127.15.208.62:46857 requested a full tablet report, sending...
I20260619 08:53:26.199693 28939 ts_manager.cc:194] Registered new tserver with Master: c43683b727c64ce9affe413c7ea39970 (127.15.208.3:44323)
I20260619 08:53:26.200206 28939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:46827
I20260619 08:53:26.209215 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:53:26.218027 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.210296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55288 (local address 127.15.208.62:46857)
0619 08:53:26.210463 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:26.210465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:26.210505 (+ 40us) server_negotiation.cc:407] Connection header received
0619 08:53:26.210585 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:26.210585 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:26.210597 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:26.210663 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:26.211322 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.212115 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.213119 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.213436 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.213506 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.214950 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:26.214951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:26.214952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:26.214959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:26.216708 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:26.217103 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:26.217104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:26.217105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:26.217169 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:26.217375 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:26.217376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:26.217376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:26.217509 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:26.217607 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:26.217728 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 08:53:26.217809 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
I20260619 08:53:26.220069 28939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:53:26.222432 28939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:53:26.230602 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.225906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45672 (local address 127.15.208.2:44759)
0619 08:53:26.226028 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:26.226033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:26.226172 (+ 139us) server_negotiation.cc:407] Connection header received
0619 08:53:26.226279 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:26.226281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:26.226297 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:26.226352 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:26.226659 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.227763 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.229306 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.230097 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.230146 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.230202 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:26.230480 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 08:53:26.230525 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260619 08:53:26.231072 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.225801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:44759 (local address 127.0.0.1:45672)
0619 08:53:26.226095 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:26.226105 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:26.226199 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:26.226496 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:26.226499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:26.226502 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:26.226580 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:26.226581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.227850 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.227851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:26.229201 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:26.229204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.230263 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.230264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:26.230344 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.230357 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 08:53:26.230934 (+ 577us) client_negotiation.cc:240] Negotiation successful
0619 08:53:26.230998 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":62,"threads_started":1}
I20260619 08:53:26.232012 29206 tablet_service.cc:1467] Processing CreateTablet for tablet f33d3754364743618683838046029659 (DEFAULT_TABLE table=test-table [id=1683a3a51dbe4d8d9fb6b2a39157eeda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:26.232383 29206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f33d3754364743618683838046029659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:26.234344 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.225989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:44323 (local address 127.0.0.1:59286)
0619 08:53:26.228815 (+ 2826us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:26.228826 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:26.228873 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:26.229204 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:26.229205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:26.229207 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:26.229247 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:26.229249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.231788 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.231789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:26.232846 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:26.232849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.233834 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.233835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:26.233895 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.233899 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:53:26.233962 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 08:53:26.234000 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2809,"thread_start_us":66,"threads_started":1}
I20260619 08:53:26.234836 29438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.226231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.15.208.3:44323)
0619 08:53:26.228984 (+ 2753us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:26.228989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:26.228998 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:53:26.229032 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:26.229034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:26.229048 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:26.229115 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:26.229328 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.231693 (+ 2365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.232967 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.233667 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.233737 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.233795 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:26.234720 (+ 925us) server_negotiation.cc:299] Negotiation successful
0619 08:53:26.234765 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2719,"thread_start_us":55,"threads_started":1}
I20260619 08:53:26.235523 29340 tablet_service.cc:1467] Processing CreateTablet for tablet f33d3754364743618683838046029659 (DEFAULT_TABLE table=test-table [id=1683a3a51dbe4d8d9fb6b2a39157eeda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:26.235829 29340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f33d3754364743618683838046029659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:26.236821 29433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.225559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:39177 (local address 127.0.0.1:41668)
0619 08:53:26.225717 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:26.225758 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:26.225837 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:26.233017 (+ 7180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:26.233018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:26.233021 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:26.233059 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:26.233060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.234348 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.234349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:26.235452 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:26.235454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.236534 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.236535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:26.236599 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.236603 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:53:26.236668 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 08:53:26.236709 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260619 08:53:26.236841 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.225653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41668 (local address 127.15.208.1:39177)
0619 08:53:26.232845 (+ 7192us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:26.232851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:26.232862 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:53:26.232896 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:26.232898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:26.232913 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:26.233127 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:26.233205 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.234245 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.235557 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.236441 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.236514 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.236566 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:26.236689 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 08:53:26.236764 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7152,"thread_start_us":57,"threads_started":1}
I20260619 08:53:26.237564 29072 tablet_service.cc:1467] Processing CreateTablet for tablet f33d3754364743618683838046029659 (DEFAULT_TABLE table=test-table [id=1683a3a51dbe4d8d9fb6b2a39157eeda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:26.237890 29072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f33d3754364743618683838046029659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:26.240252 29439 tablet_bootstrap.cc:492] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: Bootstrap starting.
I20260619 08:53:26.240998 29439 tablet_bootstrap.cc:654] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:26.241382 29439 log.cc:824] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:26.242628 29440 tablet_bootstrap.cc:492] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: Bootstrap starting.
I20260619 08:53:26.242864 29439 tablet_bootstrap.cc:492] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: No bootstrap required, opened a new log
I20260619 08:53:26.242950 29439 ts_tablet_manager.cc:1397] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:53:26.243400 29440 tablet_bootstrap.cc:654] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:26.243714 29440 log.cc:824] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:26.244040 29441 tablet_bootstrap.cc:492] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3: Bootstrap starting.
I20260619 08:53:26.244628 29441 tablet_bootstrap.cc:654] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:26.244951 29441 log.cc:824] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:26.245628 29441 tablet_bootstrap.cc:492] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3: No bootstrap required, opened a new log
I20260619 08:53:26.245684 29441 ts_tablet_manager.cc:1397] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:53:26.246166 29439 raft_consensus.cc:348] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54ff5e747f44f479351058aed0559f3" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 39177 } } peers { permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 } } peers { permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 } }
I20260619 08:53:26.246332 29439 raft_consensus.cc:374] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:26.246363 29439 raft_consensus.cc:729] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43683b727c64ce9affe413c7ea39970, State: Initialized, Role: FOLLOWER
I20260619 08:53:26.246490 29439 consensus_queue.cc:260] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54ff5e747f44f479351058aed0559f3" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 39177 } } peers { permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 } } peers { permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 } }
I20260619 08:53:26.246747 29439 ts_tablet_manager.cc:1428] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 08:53:26.246781 29406 heartbeater.cc:503] Master 127.15.208.62:46857 was elected leader, sending a full tablet report...
I20260619 08:53:26.247751 29441 raft_consensus.cc:348] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54ff5e747f44f479351058aed0559f3" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 39177 } } peers { permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 } } peers { permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 } }
I20260619 08:53:26.247898 29441 raft_consensus.cc:374] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:26.247933 29441 raft_consensus.cc:729] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f54ff5e747f44f479351058aed0559f3, State: Initialized, Role: FOLLOWER
I20260619 08:53:26.248035 29441 consensus_queue.cc:260] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54ff5e747f44f479351058aed0559f3" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 39177 } } peers { permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 } } peers { permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 } }
I20260619 08:53:26.248266 29441 ts_tablet_manager.cc:1428] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:53:26.248278 29138 heartbeater.cc:503] Master 127.15.208.62:46857 was elected leader, sending a full tablet report...
I20260619 08:53:26.253162 29440 tablet_bootstrap.cc:492] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: No bootstrap required, opened a new log
I20260619 08:53:26.253252 29440 ts_tablet_manager.cc:1397] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260619 08:53:26.255111 29440 raft_consensus.cc:348] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54ff5e747f44f479351058aed0559f3" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 39177 } } peers { permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 } } peers { permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 } }
I20260619 08:53:26.255260 29440 raft_consensus.cc:374] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:26.255307 29440 raft_consensus.cc:729] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9278c6f21d064a7987b7def0889079f6, State: Initialized, Role: FOLLOWER
I20260619 08:53:26.255432 29440 consensus_queue.cc:260] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54ff5e747f44f479351058aed0559f3" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 39177 } } peers { permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 } } peers { permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 } }
I20260619 08:53:26.255674 29440 ts_tablet_manager.cc:1428] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:53:26.255734 29272 heartbeater.cc:503] Master 127.15.208.62:46857 was elected leader, sending a full tablet report...
W20260619 08:53:26.266548 29273 tablet.cc:2369] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:53:26.271155 29446 raft_consensus.cc:482] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:53:26.271308 29446 raft_consensus.cc:504] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54ff5e747f44f479351058aed0559f3" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 39177 } } peers { permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 } } peers { permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 } }
I20260619 08:53:26.271709 29446 leader_election.cc:288] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9278c6f21d064a7987b7def0889079f6 (127.15.208.2:44759), c43683b727c64ce9affe413c7ea39970 (127.15.208.3:44323)
I20260619 08:53:26.275956 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.271950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:47483 (local address 127.15.208.2:44759)
0619 08:53:26.272007 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:26.272010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:26.272163 (+ 153us) server_negotiation.cc:407] Connection header received
0619 08:53:26.272266 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:26.272267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:26.272276 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:26.272305 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:26.272592 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.273381 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.274975 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.275554 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.275600 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.275609 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:26.275838 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 08:53:26.275874 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:53:26.276499 29438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.272347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:60199 (local address 127.15.208.3:44323)
0619 08:53:26.272393 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:26.272395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:26.272506 (+ 111us) server_negotiation.cc:407] Connection header received
0619 08:53:26.272619 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:26.272620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:26.272630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:26.272660 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:26.273329 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.274055 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.275459 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.276018 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.276069 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.276077 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:26.276372 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 08:53:26.276429 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:53:26.276381 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.271877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:44759 (local address 127.15.208.1:47483)
0619 08:53:26.272088 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:26.272101 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:26.272193 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:26.272435 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:26.272436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:26.272438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:26.272516 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:26.272517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.273556 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.273557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:26.274861 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:26.274863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.275695 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.275696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:26.275765 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.275768 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:53:26.276240 (+ 472us) client_negotiation.cc:240] Negotiation successful
0619 08:53:26.276284 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":6,"thread_start_us":143,"threads_started":1}
I20260619 08:53:26.276968 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.272164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:44323 (local address 127.15.208.1:60199)
0619 08:53:26.272440 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:26.272450 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:26.272535 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:26.273176 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:26.273177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:26.273179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:26.273246 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:26.273248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.274160 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.274160 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:26.275351 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:26.275354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.276183 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.276184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:26.276282 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.276286 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:53:26.276812 (+ 526us) client_negotiation.cc:240] Negotiation successful
0619 08:53:26.276862 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":66,"threads_started":1}
I20260619 08:53:26.277333 29226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33d3754364743618683838046029659" candidate_uuid: "f54ff5e747f44f479351058aed0559f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9278c6f21d064a7987b7def0889079f6" is_pre_election: true
I20260619 08:53:26.277530 29226 raft_consensus.cc:2454] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f54ff5e747f44f479351058aed0559f3 in term 0.
I20260619 08:53:26.277663 29360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33d3754364743618683838046029659" candidate_uuid: "f54ff5e747f44f479351058aed0559f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43683b727c64ce9affe413c7ea39970" is_pre_election: true
I20260619 08:53:26.277765 29025 leader_election.cc:302] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9278c6f21d064a7987b7def0889079f6, f54ff5e747f44f479351058aed0559f3; no voters:
I20260619 08:53:26.277812 29360 raft_consensus.cc:2454] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f54ff5e747f44f479351058aed0559f3 in term 0.
I20260619 08:53:26.278054 29446 raft_consensus.cc:2781] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:53:26.278118 29446 raft_consensus.cc:482] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:53:26.278144 29446 raft_consensus.cc:3037] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:26.279045 29446 raft_consensus.cc:504] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54ff5e747f44f479351058aed0559f3" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 39177 } } peers { permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 } } peers { permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 } }
I20260619 08:53:26.279347 29226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33d3754364743618683838046029659" candidate_uuid: "f54ff5e747f44f479351058aed0559f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9278c6f21d064a7987b7def0889079f6"
I20260619 08:53:26.279352 29360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33d3754364743618683838046029659" candidate_uuid: "f54ff5e747f44f479351058aed0559f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43683b727c64ce9affe413c7ea39970"
I20260619 08:53:26.279422 29226 raft_consensus.cc:3037] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:26.279428 29360 raft_consensus.cc:3037] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:26.280272 29226 raft_consensus.cc:2454] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f54ff5e747f44f479351058aed0559f3 in term 1.
I20260619 08:53:26.280330 29360 raft_consensus.cc:2454] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f54ff5e747f44f479351058aed0559f3 in term 1.
I20260619 08:53:26.280506 29027 leader_election.cc:302] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c43683b727c64ce9affe413c7ea39970, f54ff5e747f44f479351058aed0559f3; no voters:
I20260619 08:53:26.280709 29450 raft_consensus.cc:2781] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:53:26.280716 29446 leader_election.cc:288] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [CANDIDATE]: Term 1 election: Requested vote from peers 9278c6f21d064a7987b7def0889079f6 (127.15.208.2:44759), c43683b727c64ce9affe413c7ea39970 (127.15.208.3:44323)
I20260619 08:53:26.280969 29450 raft_consensus.cc:686] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [term 1 LEADER]: Becoming Leader. State: Replica: f54ff5e747f44f479351058aed0559f3, State: Running, Role: LEADER
I20260619 08:53:26.281111 29450 consensus_queue.cc:237] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54ff5e747f44f479351058aed0559f3" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 39177 } } peers { permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 } } peers { permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 } }
I20260619 08:53:26.281944 28939 catalog_manager.cc:5495] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f54ff5e747f44f479351058aed0559f3 (127.15.208.1). New cstate: current_term: 1 leader_uuid: "f54ff5e747f44f479351058aed0559f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54ff5e747f44f479351058aed0559f3" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 39177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 } health_report { overall_health: UNKNOWN } } }
I20260619 08:53:26.294072 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:26.290891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41676 (local address 127.15.208.1:39177)
0619 08:53:26.290939 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:26.290941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:26.290950 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:53:26.291020 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:26.291021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:26.291040 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:26.291120 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:53:26.291259 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.291999 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.293154 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:26.293453 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:26.293527 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:26.293707 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:53:26.293836 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:53:26.293928 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:26.293979 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 08:53:26.293999 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:53:26.295910 29226 raft_consensus.cc:1261] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6 [term 1 FOLLOWER]: Refusing update from remote peer f54ff5e747f44f479351058aed0559f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:53:26.295912 29360 raft_consensus.cc:1261] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970 [term 1 FOLLOWER]: Refusing update from remote peer f54ff5e747f44f479351058aed0559f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:53:26.296277 29451 consensus_queue.cc:1035] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9278c6f21d064a7987b7def0889079f6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 44759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:53:26.296386 29450 consensus_queue.cc:1035] T f33d3754364743618683838046029659 P f54ff5e747f44f479351058aed0559f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43683b727c64ce9affe413c7ea39970" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 44323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:53:26.300222 29454 mvcc.cc:205] Tried to move back new op lower bound from 7298495308985409536 to 7298495308928393216. Current Snapshot: MvccSnapshot[applied={T|T < 7298495308985409536}]
I20260619 08:53:26.300856 29456 mvcc.cc:205] Tried to move back new op lower bound from 7298495308985409536 to 7298495308928393216. Current Snapshot: MvccSnapshot[applied={T|T < 7298495308985409536}]
I20260619 08:53:26.301383 29455 mvcc.cc:205] Tried to move back new op lower bound from 7298495308985409536 to 7298495308928393216. Current Snapshot: MvccSnapshot[applied={T|T < 7298495308985409536}]
I20260619 08:53:26.304581 28940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:53:26.304682 28940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:53:26.305538 28940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31b094197162448597e78b3dde583f24: Sending DeleteTablet for 3 replicas of tablet f33d3754364743618683838046029659
I20260619 08:53:26.306053 29206 tablet_service.cc:1514] Processing DeleteTablet for tablet f33d3754364743618683838046029659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:26 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:45672
I20260619 08:53:26.306118 29072 tablet_service.cc:1514] Processing DeleteTablet for tablet f33d3754364743618683838046029659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:26 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:41668
I20260619 08:53:26.306267 29340 tablet_service.cc:1514] Processing DeleteTablet for tablet f33d3754364743618683838046029659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:26 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:59286
I20260619 08:53:26.306911 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 28978
I20260619 08:53:26.308845 29465 tablet_replica.cc:330] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: stopping tablet replica
I20260619 08:53:26.309008 29465 raft_consensus.cc:2229] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:53:26.309190 29465 raft_consensus.cc:2258] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:26.309388 29463 tablet_replica.cc:330] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: stopping tablet replica
I20260619 08:53:26.309525 29463 raft_consensus.cc:2229] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:53:26.309661 29463 raft_consensus.cc:2258] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:26.309744 29465 ts_tablet_manager.cc:1905] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:53:26.310199 29463 ts_tablet_manager.cc:1905] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:53:26.311697 29465 ts_tablet_manager.cc:1918] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:53:26.311830 29463 ts_tablet_manager.cc:1918] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:53:26.311969 29463 log.cc:1196] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/f33d3754364743618683838046029659
I20260619 08:53:26.312332 29463 ts_tablet_manager.cc:1939] T f33d3754364743618683838046029659 P 9278c6f21d064a7987b7def0889079f6: Deleting consensus metadata
I20260619 08:53:26.312793 28924 catalog_manager.cc:4841] TS 9278c6f21d064a7987b7def0889079f6 (127.15.208.2:44759): tablet f33d3754364743618683838046029659 (table test-table [id=1683a3a51dbe4d8d9fb6b2a39157eeda]) successfully deleted
W20260619 08:53:26.312987 28924 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:39177 (error 108)
I20260619 08:53:26.313659 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 29142
I20260619 08:53:26.314284 29465 log.cc:1196] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/f33d3754364743618683838046029659
I20260619 08:53:26.314592 29465 ts_tablet_manager.cc:1939] T f33d3754364743618683838046029659 P c43683b727c64ce9affe413c7ea39970: Deleting consensus metadata
I20260619 08:53:26.314991 28926 catalog_manager.cc:4841] TS c43683b727c64ce9affe413c7ea39970 (127.15.208.3:44323): tablet f33d3754364743618683838046029659 (table test-table [id=1683a3a51dbe4d8d9fb6b2a39157eeda]) successfully deleted
I20260619 08:53:26.319172 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 29275
I20260619 08:53:26.327608 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 28907
I20260619 08:53:26.334359 16192 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 08:53:26.668449 16192 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 08:53:26.668537 16192 mini_ranger.cc:66] Stopping Ranger...
I20260619 08:53:27.012892 16192 mini_ranger.cc:68] Stopped Ranger
2026-06-19 08:53:27.013 UTC [27974] LOG: received smart shutdown request
2026-06-19 08:53:27.016 UTC [27974] LOG: background worker "logical replication launcher" (PID 27983) exited with exit code 1
2026-06-19 08:53:27.016 UTC [27978] LOG: shutting down
2026-06-19 08:53:27.167 UTC [27974] LOG: database system is shut down
2026-06-19T08:53:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47629 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:53:27 dist-test-slave-p0h9 krb5kdc[29475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:53:27 dist-test-slave-p0h9 krb5kdc[29475](info): set up 2 sockets
Jun 19 08:53:27 dist-test-slave-p0h9 krb5kdc[29475](info): commencing operation
krb5kdc: starting...
W20260619 08:53:28.337152 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:53:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:53:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:53:28.457192 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:45169
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:45169,127.15.208.61:34781,127.15.208.60:41377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:28.536929 29488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:28.537142 29488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:28.537163 29488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:28.538326 29488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:53:28.538367 29488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:28.538381 29488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:28.538417 29488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:53:28.538447 29488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:53:28.539898 29488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:45169,127.15.208.61:34781,127.15.208.60:41377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:45169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:28.540144 29488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:28.540359 29488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:28.545154 29496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:28.545141 29493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:28.545141 29494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:28.545444 29488 server_base.cc:1034] running on GCE node
I20260619 08:53:28.545719 29488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:28.546000 29488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:28.547140 29488 hybrid_clock.cc:648] HybridClock initialized: now 1781859208547123 us; error 36 us; skew 500 ppm
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:28.549547 29488 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:53:28.549994 29488 webserver.cc:460] Webserver started at http://127.15.208.62:40639/ using document root <none> and password file <none>
I20260619 08:53:28.550199 29488 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:28.550259 29488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:28.550396 29488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:28.551519 29488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "8755e1bbb14b4d25bdc0e27e18375ee4"
format_stamp: "Formatted at 2026-06-19 08:53:28 on dist-test-slave-p0h9"
server_key: "fb356b3953fe02cd2eef5720cf851ab4"
server_key_iv: "f60f522a9068ccbce52a3d4d109d33cf"
server_key_version: "encryptionkey@0"
I20260619 08:53:28.551910 29488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "8755e1bbb14b4d25bdc0e27e18375ee4"
format_stamp: "Formatted at 2026-06-19 08:53:28 on dist-test-slave-p0h9"
server_key: "fb356b3953fe02cd2eef5720cf851ab4"
server_key_iv: "f60f522a9068ccbce52a3d4d109d33cf"
server_key_version: "encryptionkey@0"
I20260619 08:53:28.553309 29488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:53:28.553947 29503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:28.554131 29488 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:28.554222 29488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "8755e1bbb14b4d25bdc0e27e18375ee4"
format_stamp: "Formatted at 2026-06-19 08:53:28 on dist-test-slave-p0h9"
server_key: "fb356b3953fe02cd2eef5720cf851ab4"
server_key_iv: "f60f522a9068ccbce52a3d4d109d33cf"
server_key_version: "encryptionkey@0"
I20260619 08:53:28.554299 29488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:28.569811 29488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:28.571736 29488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:28.571897 29488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:28.576052 29488 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:45169
I20260619 08:53:28.576085 29555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:45169 every 8 connection(s)
I20260619 08:53:28.576443 29488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:53:28.577394 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:28.578902 29556 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:53:28.581980 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 29488
I20260619 08:53:28.582105 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:53:28.582284 16192 external_mini_cluster.cc:1428] Setting key d11f411379d428e704c57d0ae5af309e
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:28.588881 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.579573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50664 (local address 127.15.208.62:45169)
0619 08:53:28.579714 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.579727 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.579818 (+ 91us) server_negotiation.cc:407] Connection header received
0619 08:53:28.580298 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.580299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.580322 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.580404 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.581003 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.581792 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.582862 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.583180 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.583297 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.585689 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.585696 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.585696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.585716 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.587252 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.587704 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.587706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.587706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.587780 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.588041 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.588042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.588042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.588234 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.588291 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.588529 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.588651 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260619 08:53:28.588953 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.579534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:45169 (local address 127.0.0.1:50664)
0619 08:53:28.579697 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.579735 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:28.580178 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:28.580428 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:28.580431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:28.580696 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:28.580914 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.580918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.581907 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.581908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.582769 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.582771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.583266 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.583267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.583332 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.583869 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:28.583878 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:28.585604 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:28.587336 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.587344 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.587346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.587618 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.587847 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.587848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.587848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.587968 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.588300 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:28.588308 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:28.588391 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 08:53:28.588503 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:53:28.588775 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":71,"thread_start_us":85,"threads_started":1}
I20260619 08:53:28.589726 29556 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.208.61@KRBTEST.COM; defaulting to no policy
I20260619 08:53:28.590070 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.589944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34781 (local address 127.0.0.1:57634)
0619 08:53:28.589991 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.590044 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:34781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 08:53:28.590219 29504 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.208.61:34781: connect: Connection refused (error 111)
I20260619 08:53:28.590634 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.590560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34781 (local address 127.0.0.1:57636)
0619 08:53:28.590594 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.590619 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:34781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 08:53:28.590745 29556 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.61:34781: Network error: Client connection negotiation failed: client connection to 127.15.208.61:34781: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab.
Entry for principal kudu/127.15.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab.
I20260619 08:53:28.605543 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.61:34781
--webserver_interface=127.15.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab
--principal=kudu/127.15.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:45169,127.15.208.61:34781,127.15.208.60:41377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:53:28.640002 29556 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } attempt: 1
I20260619 08:53:28.640641 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.640471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34781 (local address 127.0.0.1:57652)
0619 08:53:28.640538 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.640593 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:34781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 08:53:28.641067 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.640937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34781 (local address 127.0.0.1:57666)
0619 08:53:28.640993 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.641031 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:34781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:53:28.641214 29556 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.61:34781: Network error: Client connection negotiation failed: client connection to 127.15.208.61:34781: connect: Connection refused (error 111)
W20260619 08:53:28.686055 29563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:28.686273 29563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:28.686295 29563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:28.687398 29563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:53:28.687446 29563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:28.687462 29563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:28.687508 29563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:53:28.687529 29563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:53:28.689134 29563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:45169,127.15.208.61:34781,127.15.208.60:41377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab
--principal=kudu/127.15.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.61:34781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:28.689371 29563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:28.689592 29563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:28.694173 29571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:28.694412 29563 server_base.cc:1034] running on GCE node
W20260619 08:53:28.694188 29568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:28.694175 29569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:28.694746 29563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:28.695017 29563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:28.696159 29563 hybrid_clock.cc:648] HybridClock initialized: now 1781859208696149 us; error 30 us; skew 500 ppm
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:28.698864 29563 init.cc:377] Logged in from keytab as kudu/127.15.208.61@KRBTEST.COM (short username kudu)
I20260619 08:53:28.699322 29563 webserver.cc:460] Webserver started at http://127.15.208.61:33741/ using document root <none> and password file <none>
I20260619 08:53:28.699560 29563 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:28.699605 29563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:28.699741 29563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:28.700842 29563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/data/instance:
uuid: "aa3fcbc77c394d8e9dd948a6a367e481"
format_stamp: "Formatted at 2026-06-19 08:53:28 on dist-test-slave-p0h9"
server_key: "8ad6e734b847b515cffd5717884cf64a"
server_key_iv: "1bfb3c0a800683695d0cb09b27cabbb4"
server_key_version: "encryptionkey@0"
I20260619 08:53:28.701196 29563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/wal/instance:
uuid: "aa3fcbc77c394d8e9dd948a6a367e481"
format_stamp: "Formatted at 2026-06-19 08:53:28 on dist-test-slave-p0h9"
server_key: "8ad6e734b847b515cffd5717884cf64a"
server_key_iv: "1bfb3c0a800683695d0cb09b27cabbb4"
server_key_version: "encryptionkey@0"
I20260619 08:53:28.702533 29563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:28.703315 29578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:28.703547 29563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:28.703647 29563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/wal
uuid: "aa3fcbc77c394d8e9dd948a6a367e481"
format_stamp: "Formatted at 2026-06-19 08:53:28 on dist-test-slave-p0h9"
server_key: "8ad6e734b847b515cffd5717884cf64a"
server_key_iv: "1bfb3c0a800683695d0cb09b27cabbb4"
server_key_version: "encryptionkey@0"
I20260619 08:53:28.703717 29563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:28.709367 29556 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } attempt: 2
I20260619 08:53:28.709820 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.709690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34781 (local address 127.0.0.1:57680)
0619 08:53:28.709746 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.709785 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:34781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 08:53:28.710224 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.710119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34781 (local address 127.0.0.1:57692)
0619 08:53:28.710165 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.710199 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:34781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:53:28.710336 29556 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.61:34781: Network error: Client connection negotiation failed: client connection to 127.15.208.61:34781: connect: Connection refused (error 111)
I20260619 08:53:28.721714 29563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:28.724146 29563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:28.724303 29563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:28.728281 29563 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.61:34781
I20260619 08:53:28.728333 29630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.61:34781 every 8 connection(s)
I20260619 08:53:28.728706 29563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
I20260619 08:53:28.729712 29631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:28.730021 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 29563
I20260619 08:53:28.730146 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-1/wal/instance
I20260619 08:53:28.730341 16192 external_mini_cluster.cc:1428] Setting key a0fccd1e926d9f3fe5d77d3da266dc60
I20260619 08:53:28.731601 29631 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.208.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.60@KRBTEST.COM" created.
I20260619 08:53:28.740782 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.732563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50672 (local address 127.15.208.62:45169)
0619 08:53:28.732639 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.732641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.732845 (+ 204us) server_negotiation.cc:407] Connection header received
0619 08:53:28.733346 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.733348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.733364 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.733415 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.734091 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.734581 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.735563 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.735775 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.735848 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.738091 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.738093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.738093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.738101 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.739405 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.739803 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.739805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.739805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.739866 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.740108 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.740109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.740110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.740230 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.740305 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.740580 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.740664 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:53:28.740995 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.732498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:45169 (local address 127.0.0.1:50672)
0619 08:53:28.732675 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.732715 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:28.733236 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:28.733493 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:28.733497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:28.733739 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:28.733993 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.733998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.734678 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.734680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.735471 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.735474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.735868 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.735869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.736024 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.736533 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:28.736541 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:28.737984 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:28.739506 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.739513 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.739515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.739730 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.739940 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.739941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.739941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.740042 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.740296 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:28.740303 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:28.740438 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 08:53:28.740559 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 08:53:28.740718 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260619 08:53:28.741678 29631 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
I20260619 08:53:28.748668 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.741942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34781 (local address 127.0.0.1:57706)
0619 08:53:28.741992 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.742002 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:28.742043 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:28.742301 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:28.742302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:28.742439 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:28.742497 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.742498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.743356 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.743357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.743850 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.743851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.744322 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.744322 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.744395 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.744870 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:28.744874 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:28.745964 (+ 1090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:28.747647 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.747648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.747648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.747826 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.748037 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.748037 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.748038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.748089 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.748352 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:28.748353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:28.748389 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 08:53:28.748467 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:53:28.748560 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:53:28.748862 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.742029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.15.208.61:34781)
0619 08:53:28.742178 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.742182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.742197 (+ 15us) server_negotiation.cc:407] Connection header received
0619 08:53:28.742213 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.742214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.742224 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.742302 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.742571 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.743253 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.743929 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.744236 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.744322 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.746046 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.746054 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.746055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.746061 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.747550 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.747912 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.747915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.747915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.747975 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.748153 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.748154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.748154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.748287 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.748373 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.748495 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.748627 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260619 08:53:28.749405 29631 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:53:28.749845 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.749746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:47900)
0619 08:53:28.749787 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.749827 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 08:53:28.750007 29579 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
I20260619 08:53:28.750488 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.750391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:47904)
0619 08:53:28.750420 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.750443 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260619 08:53:28.750613 29631 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:41377: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab.
Entry for principal kudu/127.15.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab.
I20260619 08:53:28.755399 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.60:41377
--webserver_interface=127.15.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab
--principal=kudu/127.15.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:45169,127.15.208.61:34781,127.15.208.60:41377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:53:28.799815 29631 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } attempt: 1
I20260619 08:53:28.800364 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.800230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:47920)
0619 08:53:28.800290 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.800334 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:53:28.800809 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.800681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:47924)
0619 08:53:28.800719 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.800785 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 08:53:28.800910 29631 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:41377: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
I20260619 08:53:28.801515 29556 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } attempt: 3
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
I20260619 08:53:28.808822 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.801874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57722 (local address 127.15.208.61:34781)
0619 08:53:28.801912 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.801915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.801928 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:53:28.801975 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.801976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.801986 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.802049 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.802411 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.802922 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.803887 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.804101 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.804160 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.806249 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.806250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.806251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.806258 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.807686 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.808021 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.808022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.808022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.808076 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.808277 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.808278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.808278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.808384 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.808476 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.808623 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.808697 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:53:28.808823 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.801817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34781 (local address 127.0.0.1:57722)
0619 08:53:28.801868 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.801878 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:28.801919 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:28.802071 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:28.802072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:28.802258 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:28.802332 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.802334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.803006 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.803008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.803804 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.803806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.804173 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.804173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.804250 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.804710 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:28.804716 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:28.806157 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:28.807765 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.807766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.807767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.807941 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.808139 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.808140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.808140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.808203 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.808471 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:28.808472 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:28.808517 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:53:28.808599 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:53:28.808697 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:53:28.809208 29556 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:53:28.809535 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.809420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:47938)
0619 08:53:28.809484 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.809513 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 08:53:28.809836 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.809784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:47954)
0619 08:53:28.809806 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.809825 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260619 08:53:28.809926 29556 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:41377: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
W20260619 08:53:28.838670 29638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:28.838896 29638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:28.838933 29638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:28.840167 29638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:53:28.840230 29638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:28.840255 29638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:28.840284 29638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:53:28.840314 29638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:53:28.841051 29556 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } attempt: 1
I20260619 08:53:28.841476 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.841337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:47966)
0619 08:53:28.841410 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.841450 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 08:53:28.841830 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.841715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:47976)
0619 08:53:28.841757 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.841812 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 08:53:28.841907 29556 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:41377: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
I20260619 08:53:28.842058 29638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:45169,127.15.208.61:34781,127.15.208.60:41377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab
--principal=kudu/127.15.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.60:41377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:28.842329 29638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:28.842552 29638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:28.852254 29646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:28.852283 29644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:28.852254 29643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:28.852475 29638 server_base.cc:1034] running on GCE node
I20260619 08:53:28.852777 29638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:28.853091 29638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:28.854275 29638 hybrid_clock.cc:648] HybridClock initialized: now 1781859208854259 us; error 31 us; skew 500 ppm
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:28.856868 29638 init.cc:377] Logged in from keytab as kudu/127.15.208.60@KRBTEST.COM (short username kudu)
I20260619 08:53:28.857328 29638 webserver.cc:460] Webserver started at http://127.15.208.60:45471/ using document root <none> and password file <none>
I20260619 08:53:28.857513 29638 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:28.857553 29638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:28.857646 29638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:28.858705 29638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/data/instance:
uuid: "8eb76173ddd844989c05c1acdfcd340d"
format_stamp: "Formatted at 2026-06-19 08:53:28 on dist-test-slave-p0h9"
server_key: "c1fe9765b59c5b406e55c22c6893c058"
server_key_iv: "9619c6d8b70c2f555d0c0b307d05ab37"
server_key_version: "encryptionkey@0"
I20260619 08:53:28.859000 29638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/wal/instance:
uuid: "8eb76173ddd844989c05c1acdfcd340d"
format_stamp: "Formatted at 2026-06-19 08:53:28 on dist-test-slave-p0h9"
server_key: "c1fe9765b59c5b406e55c22c6893c058"
server_key_iv: "9619c6d8b70c2f555d0c0b307d05ab37"
server_key_version: "encryptionkey@0"
I20260619 08:53:28.860299 29638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:53:28.861117 29653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:28.861295 29638 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:28.861372 29638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/wal
uuid: "8eb76173ddd844989c05c1acdfcd340d"
format_stamp: "Formatted at 2026-06-19 08:53:28 on dist-test-slave-p0h9"
server_key: "c1fe9765b59c5b406e55c22c6893c058"
server_key_iv: "9619c6d8b70c2f555d0c0b307d05ab37"
server_key_version: "encryptionkey@0"
I20260619 08:53:28.861451 29638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:28.869048 29631 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } attempt: 2
I20260619 08:53:28.869443 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.869328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:47992)
0619 08:53:28.869381 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.869417 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 08:53:28.869805 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.869723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:48008)
0619 08:53:28.869764 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.869789 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 08:53:28.869933 29631 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:41377: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41377: connect: Connection refused (error 111)
I20260619 08:53:28.871922 29638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:28.874138 29638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:28.874249 29638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:28.878095 29638 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.60:41377
I20260619 08:53:28.878124 29705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.60:41377 every 8 connection(s)
I20260619 08:53:28.878461 29638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
I20260619 08:53:28.879307 29706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:28.880178 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 29638
I20260619 08:53:28.880306 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781859057751076-16192-0/minicluster-data/master-2/wal/instance
I20260619 08:53:28.880538 16192 external_mini_cluster.cc:1428] Setting key ebd4bd4f9fb6716a447fe80642b9ea72
I20260619 08:53:28.881321 29706 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
I20260619 08:53:28.890702 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.882349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50676 (local address 127.15.208.62:45169)
0619 08:53:28.882398 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.882400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.882522 (+ 122us) server_negotiation.cc:407] Connection header received
0619 08:53:28.882597 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.882598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.882608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.882674 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.883103 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.883836 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.885068 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.885367 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.885428 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.887616 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.887618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.887618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.887626 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.889216 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.889699 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.889699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.889700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.889747 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.890010 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.890011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.890012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.890156 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.890219 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.890525 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.890600 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 08:53:28.891103 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.882257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:45169 (local address 127.0.0.1:50676)
0619 08:53:28.882430 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.882471 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:28.882536 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:28.882686 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:28.882688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:28.882903 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:28.883020 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.883025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.883915 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.883916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.884998 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.885000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.885448 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.885450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.885527 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.885959 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:28.885970 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:28.887528 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:28.889288 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.889292 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.889294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.889627 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.889808 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.889809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.889809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.889935 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.890219 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:28.890223 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:28.890379 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 08:53:28.890501 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 08:53:28.890892 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260619 08:53:28.891212 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.881075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.15.208.60:41377)
0619 08:53:28.881227 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.881236 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.881269 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:53:28.881667 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.881674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.881718 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.881824 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.882546 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.883768 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.884870 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.885269 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.885374 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.888083 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.888090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.888091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.888098 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.889825 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.890176 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.890178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.890178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.890248 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.890423 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.890424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.890424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.890593 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.890718 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.891027 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.891104 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260619 08:53:28.891673 29706 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:28.898681 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.892055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57726 (local address 127.15.208.61:34781)
0619 08:53:28.892099 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.892101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.892111 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:53:28.892137 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.892137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.892148 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.892206 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.892473 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.893105 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.893811 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.894138 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.894291 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.896162 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.896164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.896164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.896173 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.897589 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.897929 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.897930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.897930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.898008 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.898186 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.898187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.898187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.898335 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.898416 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.898510 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.898593 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:53:28.898705 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.891928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:34781 (local address 127.0.0.1:57726)
0619 08:53:28.891976 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.891986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:28.892023 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:28.892211 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:28.892212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:28.892371 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:28.892403 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.892405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.893170 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.893171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.893741 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.893743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.894394 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.894395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.894479 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.894893 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:28.894897 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:28.896080 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:28.897661 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.897662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.897663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.897856 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.898076 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.898077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.898077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.898127 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.898404 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:28.898404 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:28.898435 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 08:53:28.898498 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 08:53:28.898590 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 08:53:28.899127 29706 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:53:28.899793 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.892033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.15.208.62:45169)
0619 08:53:28.892087 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.892089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.892147 (+ 58us) server_negotiation.cc:407] Connection header received
0619 08:53:28.892212 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.892213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.892222 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.892292 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.892841 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.893323 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.894097 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.894397 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.894465 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.896827 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.896829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.896829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.896835 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.898528 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.898907 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.898909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.898909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.898966 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.899196 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.899197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.899197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.899334 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.899562 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.899617 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.899693 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
I20260619 08:53:28.905781 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.899399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:48030)
0619 08:53:28.899463 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.899472 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:28.899505 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:28.899631 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:28.899632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:28.899782 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:28.899816 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.899817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.900557 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.900558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.901113 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.901115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.901526 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.901527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.901593 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.901938 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:28.901943 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:28.903257 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:28.904779 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.904780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.904780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.904953 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.905172 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.905173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.905173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.905225 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.905461 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:28.905462 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:28.905496 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 08:53:28.905575 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:53:28.905669 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 08:53:28.905835 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.899430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.15.208.60:41377)
0619 08:53:28.899497 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.899499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.899507 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:53:28.899554 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.899555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.899564 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.899616 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.899885 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.900473 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.901189 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.901448 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.901498 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.903335 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.903336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.903336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.903343 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.904654 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.905055 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.905056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.905057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.905110 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.905290 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.905291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.905291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.905394 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.905458 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.905596 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.905719 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":48}
I20260619 08:53:28.907825 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.900511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.15.208.61:34781)
0619 08:53:28.900554 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.900556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.900647 (+ 91us) server_negotiation.cc:407] Connection header received
0619 08:53:28.900706 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.900707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.900716 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.900791 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.901238 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.901815 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.902608 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.902777 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.902830 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.905122 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.905124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.905124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.905131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.906764 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.907117 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.907118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.907118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.907176 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.907381 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.907382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.907382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.907473 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.907646 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.907714 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.907769 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:53:28.907896 29706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d: Bootstrap starting.
I20260619 08:53:28.908787 29706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:28.909107 29706 log.cc:824] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:28.909906 29706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d: No bootstrap required, opened a new log
I20260619 08:53:28.909937 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:53:28.911808 29706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } }
I20260619 08:53:28.912083 29706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:28.912184 29706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb76173ddd844989c05c1acdfcd340d, State: Initialized, Role: FOLLOWER
I20260619 08:53:28.912379 29706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } }
I20260619 08:53:28.912832 29706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:53:28.913569 29719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } } }
I20260619 08:53:28.913729 29719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:53:28.915570 29730 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:53:28.915674 29730 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:53:28.917349 29556 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } attempt: 2
I20260619 08:53:28.915268 29638 master_runner.cc:186] Error getting master registration for 127.15.208.62:45169: OK, instance_id { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" instance_seqno: 1781859208574864 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:53:28.918053 29638 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:53:28.919744 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.911773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48032 (local address 127.15.208.60:41377)
0619 08:53:28.911809 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.911811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.911818 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:53:28.911839 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.911840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.911849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.911896 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.912542 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.913282 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.915116 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.915392 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.915434 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.916898 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.916899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.916900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.916909 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.918449 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.918860 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.918862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.918862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.918916 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.919138 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.919139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.919139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.919290 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.919362 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.919581 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.919657 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:53:28.921548 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.911558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.15.208.61:34781)
0619 08:53:28.911615 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.911616 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.911623 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:53:28.911643 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.911643 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.911653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.911710 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.912509 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.913195 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.914380 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.914672 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.914739 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.916739 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.916741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.916741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.916748 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.919862 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.920594 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.920595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.920596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.920650 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.920900 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.920901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.920902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.921067 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.921159 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.921374 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.921452 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:53:28.922385 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.911235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.15.208.62:45169)
0619 08:53:28.911282 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.911283 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.911395 (+ 112us) server_negotiation.cc:407] Connection header received
0619 08:53:28.911501 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.911502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.911513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.911560 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.912551 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.913282 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.914435 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.914736 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.914800 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.919198 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.919200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.919200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.919208 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.921084 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.921479 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.921480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.921481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.921531 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.921724 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.921725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.921725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.921848 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.921928 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.922221 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.922289 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
I20260619 08:53:28.927160 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.918135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48046 (local address 127.15.208.60:41377)
0619 08:53:28.919806 (+ 1671us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.919808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.919815 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:53:28.919833 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.919834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.919844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.919908 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.920619 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.921380 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.922396 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.922661 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.922719 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.924609 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.924611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.924611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.924619 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.926135 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.926496 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.926497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.926497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.926542 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.926733 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.926734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.926734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.926827 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.926879 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.927050 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.927103 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1657,"thread_start_us":99,"threads_started":1}
I20260619 08:53:28.927215 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.917590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:48046)
0619 08:53:28.917992 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.918004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:28.918047 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:28.920239 (+ 2192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:28.920240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:28.920489 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:28.920532 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.920534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.921473 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.921474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.922225 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.922228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.922766 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.922767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.922823 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.923134 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:28.923138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:28.924535 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:28.926206 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.926207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.926208 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.926416 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.926605 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.926606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.926606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.926670 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.926899 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:28.926900 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:28.926944 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 08:53:28.927027 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:53:28.927114 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387}
I20260619 08:53:28.928856 29556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4: Bootstrap starting.
I20260619 08:53:28.929636 29556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:28.929948 29556 log.cc:824] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:28.930902 29556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4: No bootstrap required, opened a new log
I20260619 08:53:28.933149 29556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } }
I20260619 08:53:28.933305 29556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:28.933339 29556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8755e1bbb14b4d25bdc0e27e18375ee4, State: Initialized, Role: FOLLOWER
I20260619 08:53:28.933460 29556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } }
I20260619 08:53:28.933722 29734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } } }
I20260619 08:53:28.933768 29556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:53:28.933825 29734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:53:28.935629 29745 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:53:28.935731 29745 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:53:28.936331 29488 master_runner.cc:186] Error getting master registration for 127.15.208.61:34781: OK, instance_id { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" instance_seqno: 1781859208727177 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:53:28.939193 29488 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:53:28.961097 29631 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } attempt: 3
Jun 19 08:53:28 dist-test-slave-p0h9 krb5kdc[29475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859208, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
I20260619 08:53:28.969885 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.961421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41377 (local address 127.0.0.1:48048)
0619 08:53:28.961475 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:28.961486 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:28.961526 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:28.961677 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:28.961678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:28.961871 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:28.961928 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.961930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.962849 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.962850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.963679 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:28.963682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.964614 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.964615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:28.964680 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.965254 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:28.965260 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:28.966843 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:28.968760 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.968761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.968761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.968982 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.969203 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:28.969203 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:28.969204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:28.969272 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:28.969545 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:28.969545 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:28.969593 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 08:53:28.969667 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:53:28.969780 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:53:28.969885 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:28.961493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48048 (local address 127.15.208.60:41377)
0619 08:53:28.961538 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:28.961540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:28.961551 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:53:28.961581 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:28.961582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:28.961592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:28.961662 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:28.962015 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.962759 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.963766 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:28.964104 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:28.964172 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:28.966947 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:28.966949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:28.966949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:28.966958 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:28.968651 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.969071 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.969072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.969072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.969132 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:28.969337 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:28.969338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:28.969338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:28.969474 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:28.969548 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:28.969701 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 08:53:28.969793 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:53:28.972126 29631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481: Bootstrap starting.
I20260619 08:53:28.972949 29631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:28.973299 29631 log.cc:824] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:28.974228 29631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481: No bootstrap required, opened a new log
I20260619 08:53:28.976709 29631 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } }
I20260619 08:53:28.976953 29631 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:28.977043 29631 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3fcbc77c394d8e9dd948a6a367e481, State: Initialized, Role: FOLLOWER
I20260619 08:53:28.977237 29631 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } }
I20260619 08:53:28.977599 29631 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:53:28.978771 29748 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } } }
W20260619 08:53:28.979501 29759 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:53:28.979595 29759 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:53:28.981386 29563 master_runner.cc:263] No leader master found from master aa3fcbc77c394d8e9dd948a6a367e481
I20260619 08:53:28.981432 29563 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:53:28.981519 29748 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:53:29.072840 29509 tablet.cc:2369] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:53:29.124898 29659 tablet.cc:2369] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:53:29.156267 29748 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:53:29.156397 29748 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } }
I20260619 08:53:29.156711 29748 leader_election.cc:288] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8755e1bbb14b4d25bdc0e27e18375ee4 (127.15.208.62:45169), 8eb76173ddd844989c05c1acdfcd340d (127.15.208.60:41377)
I20260619 08:53:29.157042 29680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb76173ddd844989c05c1acdfcd340d" is_pre_election: true
I20260619 08:53:29.157210 29680 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa3fcbc77c394d8e9dd948a6a367e481 in term 0.
I20260619 08:53:29.157374 29530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" is_pre_election: true
I20260619 08:53:29.157497 29530 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa3fcbc77c394d8e9dd948a6a367e481 in term 0.
I20260619 08:53:29.157702 29579 leader_election.cc:302] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb76173ddd844989c05c1acdfcd340d, aa3fcbc77c394d8e9dd948a6a367e481; no voters:
I20260619 08:53:29.157874 29748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:53:29.157933 29748 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:53:29.157997 29748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:29.158771 29748 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } }
I20260619 08:53:29.158901 29748 leader_election.cc:288] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [CANDIDATE]: Term 1 election: Requested vote from peers 8755e1bbb14b4d25bdc0e27e18375ee4 (127.15.208.62:45169), 8eb76173ddd844989c05c1acdfcd340d (127.15.208.60:41377)
I20260619 08:53:29.159058 29680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb76173ddd844989c05c1acdfcd340d"
I20260619 08:53:29.159076 29530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4"
I20260619 08:53:29.159152 29680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:29.159157 29530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:29.159827 29680 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa3fcbc77c394d8e9dd948a6a367e481 in term 1.
I20260619 08:53:29.159898 29530 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa3fcbc77c394d8e9dd948a6a367e481 in term 1.
I20260619 08:53:29.160051 29579 leader_election.cc:302] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb76173ddd844989c05c1acdfcd340d, aa3fcbc77c394d8e9dd948a6a367e481; no voters:
I20260619 08:53:29.160156 29748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:53:29.160329 29748 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [term 1 LEADER]: Becoming Leader. State: Replica: aa3fcbc77c394d8e9dd948a6a367e481, State: Running, Role: LEADER
I20260619 08:53:29.160444 29748 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } }
I20260619 08:53:29.160898 29761 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3fcbc77c394d8e9dd948a6a367e481. Latest consensus state: current_term: 1 leader_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } } }
I20260619 08:53:29.160970 29761 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:29.161139 29763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:53:29.161310 29763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:53:29.162791 29680 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [term 1 FOLLOWER]: Refusing update from remote peer aa3fcbc77c394d8e9dd948a6a367e481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:53:29.162822 29530 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [term 1 FOLLOWER]: Refusing update from remote peer aa3fcbc77c394d8e9dd948a6a367e481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:53:29.163110 29761 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:53:29.163206 29748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:53:29.164430 29734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3fcbc77c394d8e9dd948a6a367e481. Latest consensus state: current_term: 1 leader_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } } }
I20260619 08:53:29.164547 29734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:53:29.164641 29719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3fcbc77c394d8e9dd948a6a367e481. Latest consensus state: current_term: 1 leader_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } } }
I20260619 08:53:29.164752 29719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:53:29.165354 29761 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } } }
I20260619 08:53:29.165432 29761 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:29.165571 29761 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } } }
I20260619 08:53:29.165647 29761 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:29.165719 29763 catalog_manager.cc:1347] Generated new cluster ID: cacbca44529041538a74ed62052f54c0
I20260619 08:53:29.165761 29763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:53:29.165762 29734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } } }
I20260619 08:53:29.165825 29734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:53:29.165870 29766 mvcc.cc:205] Tried to move back new op lower bound from 7298495320728494080 to 7298495320721698816. Current Snapshot: MvccSnapshot[applied={T|T < 7298495320728494080}]
I20260619 08:53:29.169066 29719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8755e1bbb14b4d25bdc0e27e18375ee4" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45169 } } peers { permanent_uuid: "aa3fcbc77c394d8e9dd948a6a367e481" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 34781 } } peers { permanent_uuid: "8eb76173ddd844989c05c1acdfcd340d" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41377 } } }
I20260619 08:53:29.169164 29719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:53:29.182857 29763 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:53:29.183534 29763 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:53:29.188584 29763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa3fcbc77c394d8e9dd948a6a367e481: Generated new TSK 0
I20260619 08:53:29.188827 29763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:53:29.916173 29730 catalog_manager.cc:1259] Loaded cluster ID: cacbca44529041538a74ed62052f54c0
I20260619 08:53:29.916283 29730 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d: loading cluster ID for follower catalog manager: success
I20260619 08:53:29.917235 29730 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d: acquiring CA information for follower catalog manager: success
I20260619 08:53:29.917541 29730 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8eb76173ddd844989c05c1acdfcd340d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:53:29.936198 29745 catalog_manager.cc:1259] Loaded cluster ID: cacbca44529041538a74ed62052f54c0
I20260619 08:53:29.936267 29745 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4: loading cluster ID for follower catalog manager: success
I20260619 08:53:29.937050 29745 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4: acquiring CA information for follower catalog manager: success
I20260619 08:53:29.937291 29745 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8755e1bbb14b4d25bdc0e27e18375ee4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:53:30.248081 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 29488
I20260619 08:53:30.253261 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 29563
I20260619 08:53:30.257970 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 29638
2026-06-19T08:53:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2975 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:53:30 dist-test-slave-p0h9 krb5kdc[29779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:53:30 dist-test-slave-p0h9 krb5kdc[29779](info): set up 2 sockets
Jun 19 08:53:30 dist-test-slave-p0h9 krb5kdc[29779](info): commencing operation
krb5kdc: starting...
W20260619 08:53:31.301738 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:53:31 dist-test-slave-p0h9 krb5kdc[29779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:53:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:53:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:53:31.417372 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46191
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:31.497495 29792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:31.497735 29792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:31.497773 29792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:31.498914 29792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:53:31.498977 29792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:31.499001 29792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:31.499028 29792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:53:31.499073 29792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:53:31.500521 29792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:31.500805 29792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:31.501060 29792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:31.505863 29800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:31.505877 29797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:31.506112 29798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:31.506126 29792 server_base.cc:1034] running on GCE node
I20260619 08:53:31.506397 29792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:31.506681 29792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:31.507827 29792 hybrid_clock.cc:648] HybridClock initialized: now 1781859211507798 us; error 48 us; skew 500 ppm
Jun 19 08:53:31 dist-test-slave-p0h9 krb5kdc[29779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:31.510407 29792 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:53:31.510835 29792 webserver.cc:460] Webserver started at http://127.15.208.62:32999/ using document root <none> and password file <none>
I20260619 08:53:31.511070 29792 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:31.511137 29792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:31.511292 29792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:31.512353 29792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "e4c1385112894e2eaab727090d9b8479"
format_stamp: "Formatted at 2026-06-19 08:53:31 on dist-test-slave-p0h9"
server_key: "9b5d339dcdc263c09a0180c63250ae74"
server_key_iv: "19a52d8254eba4a32d7b0af967ea686f"
server_key_version: "encryptionkey@0"
I20260619 08:53:31.512712 29792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "e4c1385112894e2eaab727090d9b8479"
format_stamp: "Formatted at 2026-06-19 08:53:31 on dist-test-slave-p0h9"
server_key: "9b5d339dcdc263c09a0180c63250ae74"
server_key_iv: "19a52d8254eba4a32d7b0af967ea686f"
server_key_version: "encryptionkey@0"
I20260619 08:53:31.514055 29792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:31.514818 29807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:31.514981 29792 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:53:31.515110 29792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "e4c1385112894e2eaab727090d9b8479"
format_stamp: "Formatted at 2026-06-19 08:53:31 on dist-test-slave-p0h9"
server_key: "9b5d339dcdc263c09a0180c63250ae74"
server_key_iv: "19a52d8254eba4a32d7b0af967ea686f"
server_key_version: "encryptionkey@0"
I20260619 08:53:31.515187 29792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:31.525972 29792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:31.527722 29792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:31.527874 29792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:31.531996 29792 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:46191
I20260619 08:53:31.532022 29859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:46191 every 8 connection(s)
I20260619 08:53:31.532387 29792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:53:31.533244 29860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:31.536643 29860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479: Bootstrap starting.
I20260619 08:53:31.537427 29860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:31.537761 29860 log.cc:824] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:31.538563 29860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479: No bootstrap required, opened a new log
I20260619 08:53:31.540990 29860 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c1385112894e2eaab727090d9b8479" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46191 } }
I20260619 08:53:31.541169 29860 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:31.541206 29860 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4c1385112894e2eaab727090d9b8479, State: Initialized, Role: FOLLOWER
I20260619 08:53:31.541352 29860 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c1385112894e2eaab727090d9b8479" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46191 } }
I20260619 08:53:31.541431 29860 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:53:31.541461 29860 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:53:31.541512 29860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:31.541723 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 29792
I20260619 08:53:31.541831 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:53:31.542119 16192 external_mini_cluster.cc:1428] Setting key b17719b7e7e849eab02baaec187a845e
I20260619 08:53:31.542559 29860 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c1385112894e2eaab727090d9b8479" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46191 } }
I20260619 08:53:31.542742 29860 leader_election.cc:302] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4c1385112894e2eaab727090d9b8479; no voters:
I20260619 08:53:31.542994 29860 leader_election.cc:288] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:53:31.543032 29865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:53:31.543217 29865 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [term 1 LEADER]: Becoming Leader. State: Replica: e4c1385112894e2eaab727090d9b8479, State: Running, Role: LEADER
I20260619 08:53:31.543304 29860 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:53:31.543366 29865 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c1385112894e2eaab727090d9b8479" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46191 } }
I20260619 08:53:31.543994 29867 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4c1385112894e2eaab727090d9b8479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c1385112894e2eaab727090d9b8479" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46191 } } }
I20260619 08:53:31.544095 29867 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:31.544456 29866 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4c1385112894e2eaab727090d9b8479. Latest consensus state: current_term: 1 leader_uuid: "e4c1385112894e2eaab727090d9b8479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c1385112894e2eaab727090d9b8479" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46191 } } }
I20260619 08:53:31.544471 29874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:53:31.545051 29874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:53:31.545122 29866 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:31.546518 29874 catalog_manager.cc:1347] Generated new cluster ID: 7f827ade7e2a490c952545104b9a181d
I20260619 08:53:31.546569 29874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:53:31 dist-test-slave-p0h9 krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:31.554078 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:31.542604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.15.208.62:46191)
0619 08:53:31.542819 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:31.542829 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:31.542863 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:53:31.543007 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:31.543016 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:31.543087 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:31.543259 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:31.544083 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:31.545572 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:31.546713 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:31.547339 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:31.547436 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:31.550232 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:31.550236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:31.550236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:31.550249 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:31.552391 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:31.552818 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:31.552822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:31.552822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:31.552937 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:31.553171 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:31.553171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:31.553172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:31.553444 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:31.553605 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:31.553703 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 08:53:31.553827 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
I20260619 08:53:31.554996 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:53:31.579138 29874 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:53:31.579931 29874 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:53:31.584821 29874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4c1385112894e2eaab727090d9b8479: Generated new TSK 0
I20260619 08:53:31.585023 29874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:53:31.609035 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 29792
2026-06-19T08:53:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1349 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:53:31 dist-test-slave-p0h9 krb5kdc[29887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:53:31 dist-test-slave-p0h9 krb5kdc[29887](info): set up 2 sockets
Jun 19 08:53:31 dist-test-slave-p0h9 krb5kdc[29887](info): commencing operation
krb5kdc: starting...
W20260619 08:53:32.646160 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:53:32 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:53:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:53:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:53:32.759161 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:38549
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:38549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:32.841172 29900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:32.841451 29900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:32.841492 29900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:32.842979 29900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:53:32.843072 29900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:32.843103 29900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:32.843135 29900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:53:32.843175 29900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:53:32.844702 29900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:38549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:38549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:32.845043 29900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:32.845273 29900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:32.850281 29908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:32.850289 29906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:32.850594 29900 server_base.cc:1034] running on GCE node
W20260619 08:53:32.850282 29905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:32.850889 29900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:32.851205 29900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:32.852375 29900 hybrid_clock.cc:648] HybridClock initialized: now 1781859212852345 us; error 48 us; skew 500 ppm
Jun 19 08:53:32 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859212, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:32.855033 29900 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:53:32.855453 29900 webserver.cc:460] Webserver started at http://127.15.208.62:38917/ using document root <none> and password file <none>
I20260619 08:53:32.855698 29900 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:32.855769 29900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:32.855921 29900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:32.857076 29900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "8e7ad409e2334281a779e7597c57fd37"
format_stamp: "Formatted at 2026-06-19 08:53:32 on dist-test-slave-p0h9"
server_key: "8b2d85e1c3369a0b806f5a952b2cf9f6"
server_key_iv: "6b41e6a55b9908bfc5a17b37f51bfb12"
server_key_version: "encryptionkey@0"
I20260619 08:53:32.857434 29900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "8e7ad409e2334281a779e7597c57fd37"
format_stamp: "Formatted at 2026-06-19 08:53:32 on dist-test-slave-p0h9"
server_key: "8b2d85e1c3369a0b806f5a952b2cf9f6"
server_key_iv: "6b41e6a55b9908bfc5a17b37f51bfb12"
server_key_version: "encryptionkey@0"
I20260619 08:53:32.859014 29900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:53:32.859820 29915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:32.860044 29900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:32.860140 29900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "8e7ad409e2334281a779e7597c57fd37"
format_stamp: "Formatted at 2026-06-19 08:53:32 on dist-test-slave-p0h9"
server_key: "8b2d85e1c3369a0b806f5a952b2cf9f6"
server_key_iv: "6b41e6a55b9908bfc5a17b37f51bfb12"
server_key_version: "encryptionkey@0"
I20260619 08:53:32.860224 29900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:32.872219 29900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:32.874171 29900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:32.874328 29900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:32.878374 29900 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:38549
I20260619 08:53:32.878408 29967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:38549 every 8 connection(s)
I20260619 08:53:32.878756 29900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:53:32.879768 29968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:32.882051 29968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Bootstrap starting.
I20260619 08:53:32.882637 29968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:32.882923 29968 log.cc:824] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:32.883569 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 29900
I20260619 08:53:32.883677 29968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: No bootstrap required, opened a new log
I20260619 08:53:32.883706 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:53:32.883900 16192 external_mini_cluster.cc:1428] Setting key a107afcbe91cb021aa4570bf0106d3dc
I20260619 08:53:32.886020 29968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:32.886199 29968 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:32.886245 29968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e7ad409e2334281a779e7597c57fd37, State: Initialized, Role: FOLLOWER
I20260619 08:53:32.886390 29968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:32.886466 29968 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:53:32.886507 29968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:53:32.886552 29968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:32.887205 29968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:32.887306 29968 leader_election.cc:302] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e7ad409e2334281a779e7597c57fd37; no voters:
I20260619 08:53:32.887445 29968 leader_election.cc:288] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:53:32.887518 29973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:53:32.887673 29968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:53:32.887697 29973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 1 LEADER]: Becoming Leader. State: Replica: 8e7ad409e2334281a779e7597c57fd37, State: Running, Role: LEADER
I20260619 08:53:32.887894 29973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:32.888240 29973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e7ad409e2334281a779e7597c57fd37. Latest consensus state: current_term: 1 leader_uuid: "8e7ad409e2334281a779e7597c57fd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } } }
I20260619 08:53:32.888459 29973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:32.888391 29975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e7ad409e2334281a779e7597c57fd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } } }
I20260619 08:53:32.888841 29975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:32.889019 29982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:53:32.889595 29982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:53:32.891353 29982 catalog_manager.cc:1347] Generated new cluster ID: fa97d6ac7c8b46c6a72773e039676602
I20260619 08:53:32.891408 29982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:53:32 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:32.895584 29972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:32.884436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.15.208.62:38549)
0619 08:53:32.884641 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:32.884651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:32.884683 (+ 32us) server_negotiation.cc:407] Connection header received
0619 08:53:32.885189 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:32.885198 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:32.885266 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:32.885413 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:32.885992 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:32.887634 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:32.888843 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:32.889445 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:32.889566 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:32.892281 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:32.892285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:32.892285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:32.892297 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:32.894036 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:32.894409 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:32.894412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:32.894413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:32.894501 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:32.894709 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:32.894710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:32.894710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:32.894933 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:32.895120 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:32.895197 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 08:53:32.895435 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260619 08:53:32.899607 29982 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:53:32.900103 29982 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:53:32.905151 29982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Generated new TSK 0
I20260619 08:53:32.905333 29982 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:53:32.935669 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38549
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:33.033470 29994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:33.033682 29994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:33.033707 29994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:33.035058 29994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:33.035107 29994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:33.035137 29994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:53:33.036594 29994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:33.036867 29994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:33.037137 29994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:33.043447 30002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:33.043478 30000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:33.043484 29999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:33.043761 29994 server_base.cc:1034] running on GCE node
I20260619 08:53:33.043998 29994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:33.044273 29994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:33.045464 29994 hybrid_clock.cc:648] HybridClock initialized: now 1781859213045441 us; error 42 us; skew 500 ppm
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:33.048307 29994 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:53:33.048874 29994 webserver.cc:460] Webserver started at http://127.15.208.1:32925/ using document root <none> and password file <none>
I20260619 08:53:33.049135 29994 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:33.049182 29994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:33.049314 29994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:33.050352 29994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "4ebb143787a742a19e5b3adb46e39556"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "670efa30942381b26024f83cc28c8516"
server_key_iv: "41014ea0dfd49a48f32bc8c908cd157f"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.050714 29994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "4ebb143787a742a19e5b3adb46e39556"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "670efa30942381b26024f83cc28c8516"
server_key_iv: "41014ea0dfd49a48f32bc8c908cd157f"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.052023 29994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:53:33.052780 30009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.052942 29994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:53:33.053051 29994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "4ebb143787a742a19e5b3adb46e39556"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "670efa30942381b26024f83cc28c8516"
server_key_iv: "41014ea0dfd49a48f32bc8c908cd157f"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.053138 29994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:33.070489 29994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:33.072863 29994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:33.073030 29994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:33.073275 29994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:33.073597 29994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:33.073650 29994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.073691 29994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:33.073719 29994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.079663 29994 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:43445
I20260619 08:53:33.079713 30122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:43445 every 8 connection(s)
I20260619 08:53:33.080019 29994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:53:33.081372 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 29994
I20260619 08:53:33.081517 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:53:33.081750 16192 external_mini_cluster.cc:1428] Setting key 4d24d01abe09ab984a0ed216e8a6af3c
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
I20260619 08:53:33.090095 29972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.080803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:56087 (local address 127.15.208.62:38549)
0619 08:53:33.080860 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.080861 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.081018 (+ 157us) server_negotiation.cc:407] Connection header received
0619 08:53:33.081523 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.081524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.081538 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.081619 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:33.082591 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.083395 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.084379 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.084782 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.084863 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.086990 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:33.086992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:33.086992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:33.087000 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:33.088615 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.089053 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.089055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.089055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.089116 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.089397 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.089398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.089398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.089532 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:33.089611 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.089890 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.089990 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:53:33.090242 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.080687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:56087)
0619 08:53:33.080880 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.080922 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.081392 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.081696 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.081699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.082303 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:33.082491 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.082496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.083485 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.083486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.084285 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.084287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.084866 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.084867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.084968 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.085512 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:33.085520 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:33.086896 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:33.088712 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.088719 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.088738 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.088966 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.089187 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.089188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.089189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.089343 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.089612 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:33.089620 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:33.089757 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.089858 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.090012 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
I20260619 08:53:33.090848 30123 heartbeater.cc:347] Connected to a master server at 127.15.208.62:38549
I20260619 08:53:33.090955 30123 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:33.091189 30123 heartbeater.cc:511] Master 127.15.208.62:38549 requested a full tablet report, sending...
I20260619 08:53:33.091718 29932 ts_manager.cc:194] Registered new tserver with Master: 4ebb143787a742a19e5b3adb46e39556 (127.15.208.1:43445)
I20260619 08:53:33.092439 29932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:56087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:53:33.105441 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38549
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:33.186810 30128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:33.187036 30128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:33.187074 30128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:33.188313 30128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:33.188373 30128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:33.188423 30128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:53:33.190107 30128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:33.190371 30128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:33.190615 30128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:33.195878 30134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:33.195933 30133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:33.195964 30128 server_base.cc:1034] running on GCE node
W20260619 08:53:33.195887 30136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:33.196311 30128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:33.196595 30128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:33.197782 30128 hybrid_clock.cc:648] HybridClock initialized: now 1781859213197769 us; error 25 us; skew 500 ppm
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:33.200182 30128 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:53:33.200553 30128 webserver.cc:460] Webserver started at http://127.15.208.2:38229/ using document root <none> and password file <none>
I20260619 08:53:33.200778 30128 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:33.200842 30128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:33.200932 30128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:33.201905 30128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "7e3611422a514e6bad8b5e873c01ae5d"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "407eee8b7588541b049d49b87ce9121c"
server_key_iv: "ad6035933b58045e55a2db24d85d4c3c"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.202204 30128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "7e3611422a514e6bad8b5e873c01ae5d"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "407eee8b7588541b049d49b87ce9121c"
server_key_iv: "ad6035933b58045e55a2db24d85d4c3c"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.203462 30128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:53:33.204144 30143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.204301 30128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:53:33.204391 30128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "7e3611422a514e6bad8b5e873c01ae5d"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "407eee8b7588541b049d49b87ce9121c"
server_key_iv: "ad6035933b58045e55a2db24d85d4c3c"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.204447 30128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:33.223467 30128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:33.225328 30128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:33.225438 30128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:33.225620 30128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:33.225867 30128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:33.225896 30128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.225917 30128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:33.225929 30128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.231679 30128 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:34759
I20260619 08:53:33.231765 30256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:34759 every 8 connection(s)
I20260619 08:53:33.232088 30128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:33.240058 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 30128
I20260619 08:53:33.240181 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:53:33.240409 16192 external_mini_cluster.cc:1428] Setting key 6a54c4a15fa27e312eb7639256c33836
I20260619 08:53:33.241299 29972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.232814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:41165 (local address 127.15.208.62:38549)
0619 08:53:33.232858 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.232859 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.233118 (+ 259us) server_negotiation.cc:407] Connection header received
0619 08:53:33.233694 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.233695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.233708 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.233756 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:33.234687 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.235181 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.236204 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.236404 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.236492 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.238601 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:33.238602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:33.238602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:33.238609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:33.239756 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.240254 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.240255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.240256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.240320 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.240619 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.240620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.240620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.240788 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:33.240851 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.241149 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.241229 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:53:33.241623 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.232817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:41165)
0619 08:53:33.232996 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.233031 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.233578 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.233823 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.233827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.234384 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:33.234580 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.234584 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.235278 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.235279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.236097 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.236099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.236522 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.236524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.236638 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.237132 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:33.237140 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:33.238515 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:33.239822 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.239828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.239830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.240182 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.240387 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.240388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.240388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.240527 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.240857 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:33.240862 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:33.240976 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.241115 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.241294 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260619 08:53:33.242087 30257 heartbeater.cc:347] Connected to a master server at 127.15.208.62:38549
I20260619 08:53:33.242194 30257 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:33.242362 30257 heartbeater.cc:511] Master 127.15.208.62:38549 requested a full tablet report, sending...
I20260619 08:53:33.242782 29932 ts_manager.cc:194] Registered new tserver with Master: 7e3611422a514e6bad8b5e873c01ae5d (127.15.208.2:34759)
I20260619 08:53:33.243321 29932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:41165
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:53:33.263397 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38549
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:33.345309 30262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:33.345543 30262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:33.345584 30262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:33.346724 30262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:33.346789 30262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:33.346832 30262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:53:33.348270 30262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:33.348541 30262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:33.348791 30262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:33.353595 30267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:33.353575 30270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:33.353574 30268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:33.353864 30262 server_base.cc:1034] running on GCE node
I20260619 08:53:33.354076 30262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:33.354312 30262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:33.355451 30262 hybrid_clock.cc:648] HybridClock initialized: now 1781859213355431 us; error 37 us; skew 500 ppm
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:33.357957 30262 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:53:33.358376 30262 webserver.cc:460] Webserver started at http://127.15.208.3:44997/ using document root <none> and password file <none>
I20260619 08:53:33.358628 30262 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:33.358696 30262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:33.358844 30262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:33.359880 30262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "97a29f119d634665b64164dce5763aa9"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "88a64d88e27567cd2f858ccbb0d59242"
server_key_iv: "09a456128fc538d9afe9546de8ef7533"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.360252 30262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "97a29f119d634665b64164dce5763aa9"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "88a64d88e27567cd2f858ccbb0d59242"
server_key_iv: "09a456128fc538d9afe9546de8ef7533"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.361614 30262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:53:33.362304 30277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.362469 30262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.362579 30262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "97a29f119d634665b64164dce5763aa9"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "88a64d88e27567cd2f858ccbb0d59242"
server_key_iv: "09a456128fc538d9afe9546de8ef7533"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.362650 30262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:33.383906 30262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:33.385841 30262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:33.386018 30262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:33.386260 30262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:33.386577 30262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:33.386633 30262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.386678 30262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:33.386706 30262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.392884 30262 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:38145
I20260619 08:53:33.392956 30390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:38145 every 8 connection(s)
I20260619 08:53:33.393301 30262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:53:33.398252 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 30262
I20260619 08:53:33.398360 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:53:33.398559 16192 external_mini_cluster.cc:1428] Setting key a28c67a2c85f4de705afa6e19affb868
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:33.403002 29972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.394130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:55209 (local address 127.15.208.62:38549)
0619 08:53:33.394176 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.394178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.394483 (+ 305us) server_negotiation.cc:407] Connection header received
0619 08:53:33.395009 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.395011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.395024 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.395072 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:33.395998 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.396503 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.397579 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.397784 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.397868 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.400329 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:33.400331 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:33.400331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:33.400338 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:33.401634 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.402031 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.402032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.402033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.402096 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.402355 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.402355 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.402356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.402470 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:33.402560 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.402834 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.402911 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:53:33.403224 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.394131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:55209)
0619 08:53:33.394316 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.394389 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.394900 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.395153 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.395157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.395714 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:33.395903 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.395908 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.396606 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.396608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.397472 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.397474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.397878 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.397879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.398043 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.398647 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:33.398658 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:33.400238 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:33.401727 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.401733 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.401736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.401951 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.402164 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.402165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.402165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.402280 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.402554 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:33.402560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:33.402699 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.402804 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.402960 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260619 08:53:33.403690 30391 heartbeater.cc:347] Connected to a master server at 127.15.208.62:38549
I20260619 08:53:33.403795 30391 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:33.403973 30391 heartbeater.cc:511] Master 127.15.208.62:38549 requested a full tablet report, sending...
I20260619 08:53:33.404383 29932 ts_manager.cc:194] Registered new tserver with Master: 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145)
I20260619 08:53:33.404942 29932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:55209
I20260619 08:53:33.410331 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:53:33.417835 29972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.411653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34264 (local address 127.15.208.62:38549)
0619 08:53:33.411730 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.411733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.411821 (+ 88us) server_negotiation.cc:407] Connection header received
0619 08:53:33.411914 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.411915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.411925 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.411999 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:33.412558 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.413126 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.413843 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.414047 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.414131 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.415356 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:33.415358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:33.415358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:33.415365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:33.416581 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.416996 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.416997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.416997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.417061 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.417272 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.417273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.417273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.417425 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:33.417534 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.417648 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.417734 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 08:53:33.419374 29932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:53:33.421054 29932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:53:33.427680 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.423575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.15.208.2:34759)
0619 08:53:33.423758 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.423763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.423817 (+ 54us) server_negotiation.cc:407] Connection header received
0619 08:53:33.423938 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.423940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.423959 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.424056 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:33.424301 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.425173 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.426684 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.427242 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.427300 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.427375 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.427545 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.427587 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":81,"threads_started":1}
I20260619 08:53:33.427703 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.423315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43445 (local address 127.0.0.1:41502)
0619 08:53:33.423944 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.423953 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.424006 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.424421 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.424421 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.424424 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:33.424455 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.424456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.425690 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.425691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.426705 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.426708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.427370 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.427371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.427444 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.427449 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.427522 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.427588 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":21,"thread_start_us":108,"threads_started":1}
I20260619 08:53:33.427965 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.423563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.15.208.1:43445)
0619 08:53:33.423840 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.423844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.424267 (+ 423us) server_negotiation.cc:407] Connection header received
0619 08:53:33.424316 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.424318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.424333 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.424408 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:33.424518 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.425587 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.426782 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.427259 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.427352 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.427414 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.427827 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.427884 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":85,"threads_started":1}
I20260619 08:53:33.428231 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.423310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:34759 (local address 127.0.0.1:43274)
0619 08:53:33.423744 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.423756 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.423847 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.424137 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.424138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.424140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:33.424222 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.424224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.425272 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.425273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.426592 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.426594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.427338 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.427339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.427431 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.427446 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.427517 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.427585 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":7,"thread_start_us":86,"threads_started":1}
I20260619 08:53:33.428758 30403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.423310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:38145 (local address 127.0.0.1:53680)
0619 08:53:33.423703 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.423741 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.423835 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.424112 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.424115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.424118 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:33.424185 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.424187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.425520 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.425521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.426592 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.426594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.428493 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.428494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.428557 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.428560 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.428610 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.428649 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":3,"thread_start_us":73,"threads_started":1}
I20260619 08:53:33.428689 30057 tablet_service.cc:1467] Processing CreateTablet for tablet 8db3482bc0fb48e3b219b90e3ae61883 (DEFAULT_TABLE table=test-table [id=385c77c2cc514768afd7cc7b57607f0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:33.429066 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.423499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53680 (local address 127.15.208.3:38145)
0619 08:53:33.423667 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.423673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.423815 (+ 142us) server_negotiation.cc:407] Connection header received
0619 08:53:33.423921 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.423924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.423939 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.424015 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:33.424244 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.425405 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.427411 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.428217 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.428402 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.428461 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.428925 (+ 464us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.428978 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260619 08:53:33.429108 30057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db3482bc0fb48e3b219b90e3ae61883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:33.429071 30191 tablet_service.cc:1467] Processing CreateTablet for tablet 8db3482bc0fb48e3b219b90e3ae61883 (DEFAULT_TABLE table=test-table [id=385c77c2cc514768afd7cc7b57607f0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:33.429415 30191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db3482bc0fb48e3b219b90e3ae61883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:33.429697 30325 tablet_service.cc:1467] Processing CreateTablet for tablet 8db3482bc0fb48e3b219b90e3ae61883 (DEFAULT_TABLE table=test-table [id=385c77c2cc514768afd7cc7b57607f0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:33.429993 30325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db3482bc0fb48e3b219b90e3ae61883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:33.431746 30408 tablet_bootstrap.cc:492] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Bootstrap starting.
I20260619 08:53:33.432394 30408 tablet_bootstrap.cc:654] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:33.432786 30409 tablet_bootstrap.cc:492] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: Bootstrap starting.
I20260619 08:53:33.433048 30408 log.cc:824] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:33.433297 30410 tablet_bootstrap.cc:492] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: Bootstrap starting.
I20260619 08:53:33.433468 30409 tablet_bootstrap.cc:654] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:33.433938 30409 log.cc:824] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:33.433974 30410 tablet_bootstrap.cc:654] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:33.434487 30410 log.cc:824] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:33.434702 30409 tablet_bootstrap.cc:492] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: No bootstrap required, opened a new log
I20260619 08:53:33.434816 30409 ts_tablet_manager.cc:1397] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:53:33.435248 30410 tablet_bootstrap.cc:492] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: No bootstrap required, opened a new log
I20260619 08:53:33.435353 30410 ts_tablet_manager.cc:1397] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:53:33.435421 30408 tablet_bootstrap.cc:492] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: No bootstrap required, opened a new log
I20260619 08:53:33.435491 30408 ts_tablet_manager.cc:1397] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 08:53:33.436924 30408 raft_consensus.cc:348] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.436947 30409 raft_consensus.cc:348] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.437083 30408 raft_consensus.cc:374] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:33.437098 30409 raft_consensus.cc:374] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:33.437127 30408 raft_consensus.cc:729] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ebb143787a742a19e5b3adb46e39556, State: Initialized, Role: FOLLOWER
I20260619 08:53:33.437134 30409 raft_consensus.cc:729] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e3611422a514e6bad8b5e873c01ae5d, State: Initialized, Role: FOLLOWER
I20260619 08:53:33.437239 30408 consensus_queue.cc:260] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.437278 30409 consensus_queue.cc:260] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.437317 30410 raft_consensus.cc:348] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.437454 30410 raft_consensus.cc:374] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:33.437490 30410 raft_consensus.cc:729] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a29f119d634665b64164dce5763aa9, State: Initialized, Role: FOLLOWER
I20260619 08:53:33.437522 30408 ts_tablet_manager.cc:1428] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:53:33.437543 30123 heartbeater.cc:503] Master 127.15.208.62:38549 was elected leader, sending a full tablet report...
I20260619 08:53:33.437590 30410 consensus_queue.cc:260] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.437870 30409 ts_tablet_manager.cc:1428] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:53:33.437887 30391 heartbeater.cc:503] Master 127.15.208.62:38549 was elected leader, sending a full tablet report...
I20260619 08:53:33.438045 30257 heartbeater.cc:503] Master 127.15.208.62:38549 was elected leader, sending a full tablet report...
I20260619 08:53:33.437870 30410 ts_tablet_manager.cc:1428] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:53:33.445546 30415 raft_consensus.cc:482] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:53:33.445659 30415 raft_consensus.cc:504] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.446002 30415 leader_election.cc:288] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145), 4ebb143787a742a19e5b3adb46e39556 (127.15.208.1:43445)
I20260619 08:53:33.448824 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.446241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:54647 (local address 127.15.208.3:38145)
0619 08:53:33.446331 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.446334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.446342 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:53:33.446367 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.446368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.446380 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.446446 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:33.446714 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.447180 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.448083 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.448461 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.448545 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.448554 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.448700 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.448756 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 08:53:33.448849 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.446169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:38145 (local address 127.15.208.2:54647)
0619 08:53:33.446230 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.446243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.446285 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.446571 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.446572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.446573 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:33.446622 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.446623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.447264 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.447265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.447992 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.447995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.448542 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.448543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.448614 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.448618 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.448691 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.448758 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 08:53:33.448874 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.446243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:35033 (local address 127.15.208.1:43445)
0619 08:53:33.446283 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.446284 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.446428 (+ 144us) server_negotiation.cc:407] Connection header received
0619 08:53:33.446481 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.446481 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.446489 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.446530 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:33.446664 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.447185 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.448104 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.448476 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.448579 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.448587 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.448768 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.448806 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:53:33.449024 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.446169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43445 (local address 127.15.208.2:35033)
0619 08:53:33.446366 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.446377 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.446436 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.446537 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.446538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.446539 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:33.446593 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.446595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.447276 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.447276 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.448008 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.448010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.448580 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.448581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.448657 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.448660 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.448767 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.448805 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":21,"thread_start_us":100,"threads_started":1}
I20260619 08:53:33.449191 30345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db3482bc0fb48e3b219b90e3ae61883" candidate_uuid: "7e3611422a514e6bad8b5e873c01ae5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a29f119d634665b64164dce5763aa9" is_pre_election: true
I20260619 08:53:33.449357 30345 raft_consensus.cc:2454] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e3611422a514e6bad8b5e873c01ae5d in term 0.
I20260619 08:53:33.449331 30077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db3482bc0fb48e3b219b90e3ae61883" candidate_uuid: "7e3611422a514e6bad8b5e873c01ae5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ebb143787a742a19e5b3adb46e39556" is_pre_election: true
I20260619 08:53:33.449519 30077 raft_consensus.cc:2454] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e3611422a514e6bad8b5e873c01ae5d in term 0.
I20260619 08:53:33.449600 30146 leader_election.cc:302] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e3611422a514e6bad8b5e873c01ae5d, 97a29f119d634665b64164dce5763aa9; no voters:
I20260619 08:53:33.449743 30415 raft_consensus.cc:2781] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:53:33.449819 30415 raft_consensus.cc:482] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:53:33.449855 30415 raft_consensus.cc:3037] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:33.450536 30415 raft_consensus.cc:504] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.450753 30415 leader_election.cc:288] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [CANDIDATE]: Term 1 election: Requested vote from peers 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145), 4ebb143787a742a19e5b3adb46e39556 (127.15.208.1:43445)
I20260619 08:53:33.450877 30345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db3482bc0fb48e3b219b90e3ae61883" candidate_uuid: "7e3611422a514e6bad8b5e873c01ae5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a29f119d634665b64164dce5763aa9"
I20260619 08:53:33.450876 30077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db3482bc0fb48e3b219b90e3ae61883" candidate_uuid: "7e3611422a514e6bad8b5e873c01ae5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ebb143787a742a19e5b3adb46e39556"
I20260619 08:53:33.450955 30345 raft_consensus.cc:3037] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:33.450976 30077 raft_consensus.cc:3037] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:33.451622 30345 raft_consensus.cc:2454] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e3611422a514e6bad8b5e873c01ae5d in term 1.
I20260619 08:53:33.451644 30077 raft_consensus.cc:2454] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e3611422a514e6bad8b5e873c01ae5d in term 1.
I20260619 08:53:33.451807 30147 leader_election.cc:302] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ebb143787a742a19e5b3adb46e39556, 7e3611422a514e6bad8b5e873c01ae5d; no voters:
I20260619 08:53:33.451956 30415 raft_consensus.cc:2781] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:53:33.452124 30415 raft_consensus.cc:686] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 LEADER]: Becoming Leader. State: Replica: 7e3611422a514e6bad8b5e873c01ae5d, State: Running, Role: LEADER
I20260619 08:53:33.452226 30415 consensus_queue.cc:237] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.452993 29931 catalog_manager.cc:5495] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e3611422a514e6bad8b5e873c01ae5d (127.15.208.2). New cstate: current_term: 1 leader_uuid: "7e3611422a514e6bad8b5e873c01ae5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } health_report { overall_health: UNKNOWN } } }
I20260619 08:53:33.463793 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.461387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.15.208.2:34759)
0619 08:53:33.461440 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.461442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.461451 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:53:33.461522 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.461530 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.461547 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.461632 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:53:33.461770 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.462219 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.462951 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.463196 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.463286 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.463447 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:53:33.463555 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:53:33.463644 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.463684 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.463702 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:53:33.465574 30077 raft_consensus.cc:1261] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Refusing update from remote peer 7e3611422a514e6bad8b5e873c01ae5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:53:33.465590 30345 raft_consensus.cc:1261] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9 [term 1 FOLLOWER]: Refusing update from remote peer 7e3611422a514e6bad8b5e873c01ae5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:53:33.465889 30418 consensus_queue.cc:1035] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:53:33.466014 30415 consensus_queue.cc:1035] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:53:33.468199 30421 mvcc.cc:205] Tried to move back new op lower bound from 7298495338352185344 to 7298495338300887040. Current Snapshot: MvccSnapshot[applied={T|T < 7298495338352185344}]
I20260619 08:53:33.468586 30422 mvcc.cc:205] Tried to move back new op lower bound from 7298495338352185344 to 7298495338300887040. Current Snapshot: MvccSnapshot[applied={T|T < 7298495338352185344 or (T in {7298495338352185344})}]
I20260619 08:53:33.468688 30423 mvcc.cc:205] Tried to move back new op lower bound from 7298495338352185344 to 7298495338300887040. Current Snapshot: MvccSnapshot[applied={T|T < 7298495338352185344}]
I20260619 08:53:33.470901 29931 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:53:33.471004 29931 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:53:33.471719 29931 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Sending DeleteTablet for 3 replicas of tablet 8db3482bc0fb48e3b219b90e3ae61883
I20260619 08:53:33.472033 30191 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db3482bc0fb48e3b219b90e3ae61883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:33 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:43274
I20260619 08:53:33.472059 30325 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db3482bc0fb48e3b219b90e3ae61883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:33 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53680
I20260619 08:53:33.472182 30057 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db3482bc0fb48e3b219b90e3ae61883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:33 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:41502
I20260619 08:53:33.472322 30430 tablet_replica.cc:330] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: stopping tablet replica
I20260619 08:53:33.472381 30432 tablet_replica.cc:330] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: stopping tablet replica
I20260619 08:53:33.472436 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 29994
I20260619 08:53:33.472510 30432 raft_consensus.cc:2229] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:53:33.472606 30432 raft_consensus.cc:2258] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:33.473645 30431 tablet_replica.cc:330] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: stopping tablet replica
I20260619 08:53:33.473757 30431 raft_consensus.cc:2229] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:53:33.473851 30431 raft_consensus.cc:2258] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:33.474263 30431 ts_tablet_manager.cc:1905] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:53:33.475627 30431 ts_tablet_manager.cc:1918] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:53:33.475713 30431 log.cc:1196] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/8db3482bc0fb48e3b219b90e3ae61883
I20260619 08:53:33.475986 30431 ts_tablet_manager.cc:1939] T 8db3482bc0fb48e3b219b90e3ae61883 P 97a29f119d634665b64164dce5763aa9: Deleting consensus metadata
I20260619 08:53:33.476385 29918 catalog_manager.cc:4841] TS 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145): tablet 8db3482bc0fb48e3b219b90e3ae61883 (table test-table [id=385c77c2cc514768afd7cc7b57607f0b]) successfully deleted
I20260619 08:53:33.472491 30430 raft_consensus.cc:2229] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:53:33.476933 30430 raft_consensus.cc:2258] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:33.477377 30430 ts_tablet_manager.cc:1905] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:53:33.478673 30430 ts_tablet_manager.cc:1918] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:53:33.478771 30430 log.cc:1196] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/8db3482bc0fb48e3b219b90e3ae61883
I20260619 08:53:33.479058 30430 ts_tablet_manager.cc:1939] T 8db3482bc0fb48e3b219b90e3ae61883 P 7e3611422a514e6bad8b5e873c01ae5d: Deleting consensus metadata
W20260619 08:53:33.479323 29919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:43445 (error 108)
I20260619 08:53:33.479486 29916 catalog_manager.cc:4841] TS 7e3611422a514e6bad8b5e873c01ae5d (127.15.208.2:34759): tablet 8db3482bc0fb48e3b219b90e3ae61883 (table test-table [id=385c77c2cc514768afd7cc7b57607f0b]) successfully deleted
I20260619 08:53:33.479604 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 30128
I20260619 08:53:33.480227 30403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.479781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43445 (local address 127.0.0.1:41518)
0619 08:53:33.480154 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.480188 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:43445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352}
W20260619 08:53:33.480367 29919 catalog_manager.cc:4584] TS 4ebb143787a742a19e5b3adb46e39556 (127.15.208.1:43445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8db3482bc0fb48e3b219b90e3ae61883: Network error: Client connection negotiation failed: client connection to 127.15.208.1:43445: connect: Connection refused (error 111)
I20260619 08:53:33.483798 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 30262
I20260619 08:53:33.488096 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 29900
I20260619 08:53:33.493217 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:38549
--webserver_interface=127.15.208.62
--webserver_port=38917
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:38549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:33.574128 30434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:33.574368 30434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:33.574405 30434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:33.575551 30434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:53:33.575620 30434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:33.575645 30434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:33.575665 30434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:53:33.575695 30434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:53:33.577198 30434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:38549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:38549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=38917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:33.577476 30434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:33.577709 30434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:33.585937 30441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:33.585983 30440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:33.586079 30434 server_base.cc:1034] running on GCE node
W20260619 08:53:33.585938 30443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:33.586506 30434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:33.586875 30434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:33.588020 30434 hybrid_clock.cc:648] HybridClock initialized: now 1781859213588000 us; error 36 us; skew 500 ppm
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:33.590858 30434 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:53:33.591357 30434 webserver.cc:460] Webserver started at http://127.15.208.62:38917/ using document root <none> and password file <none>
I20260619 08:53:33.591617 30434 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:33.591693 30434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:33.593151 30434 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:53:33.593796 30450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.593968 30434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:53:33.594065 30434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "8e7ad409e2334281a779e7597c57fd37"
format_stamp: "Formatted at 2026-06-19 08:53:32 on dist-test-slave-p0h9"
server_key: "8b2d85e1c3369a0b806f5a952b2cf9f6"
server_key_iv: "6b41e6a55b9908bfc5a17b37f51bfb12"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.594480 30434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:33.618397 30434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:33.620249 30434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:33.620419 30434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:33.624704 30434 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:38549
I20260619 08:53:33.624811 30502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:38549 every 8 connection(s)
I20260619 08:53:33.625190 30434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:53:33.626812 30503 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:53:33.627521 30503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Bootstrap starting.
I20260619 08:53:33.627921 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 30434
I20260619 08:53:33.628252 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:43445
--local_ip_for_outbound_sockets=127.15.208.1
--tserver_master_addrs=127.15.208.62:38549
--webserver_port=32925
--webserver_interface=127.15.208.1
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:53:33.636175 30503 log.cc:824] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:33.638144 30503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:53:33.638477 30503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Bootstrap complete.
I20260619 08:53:33.640825 30503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:33.641100 30503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e7ad409e2334281a779e7597c57fd37, State: Initialized, Role: FOLLOWER
I20260619 08:53:33.641304 30503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:33.641387 30503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:53:33.641445 30503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:53:33.641510 30503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:53:33.642586 30503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:33.642704 30503 leader_election.cc:302] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e7ad409e2334281a779e7597c57fd37; no voters:
I20260619 08:53:33.642905 30503 leader_election.cc:288] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:53:33.642940 30507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:53:33.643059 30507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 2 LEADER]: Becoming Leader. State: Replica: 8e7ad409e2334281a779e7597c57fd37, State: Running, Role: LEADER
I20260619 08:53:33.643213 30503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:53:33.643239 30507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:33.643640 30509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e7ad409e2334281a779e7597c57fd37. Latest consensus state: current_term: 2 leader_uuid: "8e7ad409e2334281a779e7597c57fd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } } }
I20260619 08:53:33.643728 30509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:33.643949 30508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8e7ad409e2334281a779e7597c57fd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } } }
I20260619 08:53:33.644029 30508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:33.644152 30514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:53:33.645061 30514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:53:33.645383 30514 catalog_manager.cc:1259] Loaded cluster ID: fa97d6ac7c8b46c6a72773e039676602
I20260619 08:53:33.645426 30514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:53:33.645660 30514 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 08:53:33.648326 30514 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 08:53:33.650208 30514 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:53:33.650414 30514 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Loaded TSK: 0
I20260619 08:53:33.650635 30514 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:53:33.730353 30505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:33.730572 30505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:33.730603 30505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:33.731783 30505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:33.731825 30505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:33.731851 30505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:53:33.733282 30505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:43445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=32925
--tserver_master_addrs=127.15.208.62:38549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:33.733505 30505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:33.733800 30505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:33.739660 30533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:33.739660 30531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:33.739881 30505 server_base.cc:1034] running on GCE node
W20260619 08:53:33.739646 30530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:33.740190 30505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:33.740449 30505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:33.741626 30505 hybrid_clock.cc:648] HybridClock initialized: now 1781859213741610 us; error 31 us; skew 500 ppm
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:33.744490 30505 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:53:33.745082 30505 webserver.cc:460] Webserver started at http://127.15.208.1:32925/ using document root <none> and password file <none>
I20260619 08:53:33.745347 30505 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:33.745431 30505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:33.747212 30505 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:53:33.747992 30540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.748200 30505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:33.748296 30505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "4ebb143787a742a19e5b3adb46e39556"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "670efa30942381b26024f83cc28c8516"
server_key_iv: "41014ea0dfd49a48f32bc8c908cd157f"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.748764 30505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:33.780987 30505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:33.783601 30505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:33.783766 30505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:33.784031 30505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:33.784556 30547 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:53:33.786056 30505 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:53:33.786119 30505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:53:33.786156 30505 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:53:33.787034 30505 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:53:33.787083 30505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:53:33.787153 30547 tablet_bootstrap.cc:492] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Bootstrap starting.
I20260619 08:53:33.794199 30505 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:43445
I20260619 08:53:33.794426 30654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:43445 every 8 connection(s)
I20260619 08:53:33.794696 30505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:53:33.798107 30547 log.cc:824] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:33.800493 30547 tablet_bootstrap.cc:492] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:53:33.800836 30547 tablet_bootstrap.cc:492] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Bootstrap complete.
I20260619 08:53:33.802057 30547 ts_tablet_manager.cc:1397] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.013s
I20260619 08:53:33.803385 30547 raft_consensus.cc:348] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.803634 30547 raft_consensus.cc:729] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ebb143787a742a19e5b3adb46e39556, State: Initialized, Role: FOLLOWER
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:33.803822 30547 consensus_queue.cc:260] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:33.804188 30547 ts_tablet_manager.cc:1428] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 08:53:33.804352 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 30505
I20260619 08:53:33.804668 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:34759
--local_ip_for_outbound_sockets=127.15.208.2
--tserver_master_addrs=127.15.208.62:38549
--webserver_port=38229
--webserver_interface=127.15.208.2
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:53:33.808391 30657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.795460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:34959)
0619 08:53:33.795750 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.795795 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.796354 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.796957 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.796961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.797383 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:33.797650 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.797657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.799375 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.799377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.800619 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.800622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.801633 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.801635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.801774 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.802391 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:33.802401 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:33.804267 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:33.806512 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.806520 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.806523 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.806838 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.807138 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.807139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.807139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.807268 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.807761 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:33.807768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:33.807921 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.808033 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.808194 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":107,"threads_started":1}
I20260619 08:53:33.808501 30658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.795542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:34959 (local address 127.15.208.62:38549)
0619 08:53:33.795834 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.795846 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.795903 (+ 57us) server_negotiation.cc:407] Connection header received
0619 08:53:33.796628 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.796638 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.796719 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.796934 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:33.797802 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.799244 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.800749 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.801411 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.801526 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.804382 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:33.804386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:33.804387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:33.804429 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:33.806399 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.806930 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.806933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.806934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.807060 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.807349 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.807350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.807350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.807654 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:33.807734 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.808114 (+ 380us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.808267 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":91,"threads_started":1}
I20260619 08:53:33.809587 30655 heartbeater.cc:347] Connected to a master server at 127.15.208.62:38549
I20260619 08:53:33.809681 30655 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:33.809870 30655 heartbeater.cc:511] Master 127.15.208.62:38549 requested a full tablet report, sending...
I20260619 08:53:33.810710 30467 ts_manager.cc:194] Registered new tserver with Master: 4ebb143787a742a19e5b3adb46e39556 (127.15.208.1:43445)
I20260619 08:53:33.812101 30467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:34959
I20260619 08:53:33.813200 30655 heartbeater.cc:503] Master 127.15.208.62:38549 was elected leader, sending a full tablet report...
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:53:33.820828 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.811723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41526 (local address 127.15.208.1:43445)
0619 08:53:33.812096 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.812101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.812109 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:53:33.812143 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.812144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.812157 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.812289 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:33.813002 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.814188 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.815437 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.815714 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.815771 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.817970 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:33.817978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:33.817978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:33.817989 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:33.819499 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.819927 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.819930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.819930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.819986 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.820197 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.820198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.820198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.820332 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:33.820412 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.820599 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.820704 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":64,"threads_started":1}
I20260619 08:53:33.820825 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.811421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43445 (local address 127.0.0.1:41526)
0619 08:53:33.811885 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.811921 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.811982 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.812360 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.812366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.812860 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:33.812916 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.812919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.814281 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.814283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.815342 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.815345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.815797 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.815798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.815887 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.816376 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:33.816382 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:33.817877 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:33.819619 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.819626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.819628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.819835 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.820065 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.820066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.820066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.820127 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.820405 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:33.820411 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:33.820462 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.820562 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.820701 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":125,"threads_started":1}
I20260619 08:53:33.821255 30589 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db3482bc0fb48e3b219b90e3ae61883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:33 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:41526
I20260619 08:53:33.821858 30664 tablet_replica.cc:330] stopping tablet replica
I20260619 08:53:33.821977 30664 raft_consensus.cc:2229] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:53:33.822059 30664 raft_consensus.cc:2258] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:33.822383 30664 ts_tablet_manager.cc:1905] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:53:33.824314 30664 ts_tablet_manager.cc:1918] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:53:33.824416 30664 log.cc:1196] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/8db3482bc0fb48e3b219b90e3ae61883
I20260619 08:53:33.824626 30664 ts_tablet_manager.cc:1939] T 8db3482bc0fb48e3b219b90e3ae61883 P 4ebb143787a742a19e5b3adb46e39556: Deleting consensus metadata
I20260619 08:53:33.825117 30454 catalog_manager.cc:4841] TS 4ebb143787a742a19e5b3adb46e39556 (127.15.208.1:43445): tablet 8db3482bc0fb48e3b219b90e3ae61883 (table test-table [id=385c77c2cc514768afd7cc7b57607f0b]) successfully deleted
W20260619 08:53:33.889475 30661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:33.889688 30661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:33.889710 30661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:33.890985 30661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:33.891029 30661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:33.891057 30661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:53:33.892472 30661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:34759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=38229
--tserver_master_addrs=127.15.208.62:38549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:33.892657 30661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:33.892926 30661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:33.897758 30672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:33.897789 30670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:33.897802 30669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:33.897970 30661 server_base.cc:1034] running on GCE node
I20260619 08:53:33.898253 30661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:33.898515 30661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:33.899689 30661 hybrid_clock.cc:648] HybridClock initialized: now 1781859213899671 us; error 33 us; skew 500 ppm
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:33.902264 30661 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:53:33.902740 30661 webserver.cc:460] Webserver started at http://127.15.208.2:38229/ using document root <none> and password file <none>
I20260619 08:53:33.902954 30661 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:33.903033 30661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:33.904356 30661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:33.905007 30679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.905180 30661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:33.905282 30661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "7e3611422a514e6bad8b5e873c01ae5d"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "407eee8b7588541b049d49b87ce9121c"
server_key_iv: "ad6035933b58045e55a2db24d85d4c3c"
server_key_version: "encryptionkey@0"
I20260619 08:53:33.905651 30661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:33.946848 30661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:33.949326 30661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:33.949493 30661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:33.949743 30661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:33.950050 30661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:33.950103 30661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.950143 30661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:33.950167 30661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:33.956171 30661 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:34759
I20260619 08:53:33.956223 30792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:34759 every 8 connection(s)
I20260619 08:53:33.956609 30661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:53:33.961086 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 30661
I20260619 08:53:33.961397 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:38145
--local_ip_for_outbound_sockets=127.15.208.3
--tserver_master_addrs=127.15.208.62:38549
--webserver_port=44997
--webserver_interface=127.15.208.3
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:53:33 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:33.966586 30658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.957394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:56153 (local address 127.15.208.62:38549)
0619 08:53:33.957454 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:33.957456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:33.957714 (+ 258us) server_negotiation.cc:407] Connection header received
0619 08:53:33.958183 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:33.958184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:33.958192 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:33.958246 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:33.959179 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.959667 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.960606 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.960864 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.960972 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.963495 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:33.963497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:33.963497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:33.963505 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:33.965102 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.965508 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.965509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.965510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.965581 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:33.965846 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:33.965847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:33.965847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:33.965988 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:33.966050 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:33.966443 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 08:53:33.966514 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:53:33.966912 30795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:33.957374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:56153)
0619 08:53:33.957571 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:33.957615 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:33.958100 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:33.958307 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:33.958310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:33.958891 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:33.959093 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.959099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.959746 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.959747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.960507 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:33.960508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:33.960984 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:33.960985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:33.961116 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:33.961703 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:33.961711 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:33.963376 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:33.965192 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.965198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.965200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.965430 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.965658 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:33.965658 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:33.965659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:33.965772 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:33.966057 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:33.966065 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:33.966282 (+ 217us) client_negotiation.cc:769] Sending connection context
0619 08:53:33.966427 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 08:53:33.966630 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":114,"threads_started":1}
I20260619 08:53:33.967377 30793 heartbeater.cc:347] Connected to a master server at 127.15.208.62:38549
I20260619 08:53:33.967485 30793 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:33.967672 30793 heartbeater.cc:511] Master 127.15.208.62:38549 requested a full tablet report, sending...
I20260619 08:53:33.968089 30467 ts_manager.cc:194] Registered new tserver with Master: 7e3611422a514e6bad8b5e873c01ae5d (127.15.208.2:34759)
I20260619 08:53:33.968472 30467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:56153
W20260619 08:53:34.042608 30796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:34.042805 30796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:34.042827 30796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:34.044138 30796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:34.044180 30796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:34.044229 30796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:53:34.045696 30796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:38145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=44997
--tserver_master_addrs=127.15.208.62:38549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:34.045883 30796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:34.046129 30796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:34.050868 30802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:34.050899 30804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:34.051010 30796 server_base.cc:1034] running on GCE node
W20260619 08:53:34.050877 30801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:34.051296 30796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:34.051517 30796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:34.052662 30796 hybrid_clock.cc:648] HybridClock initialized: now 1781859214052654 us; error 29 us; skew 500 ppm
Jun 19 08:53:34 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859214, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:34.055204 30796 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:53:34.055687 30796 webserver.cc:460] Webserver started at http://127.15.208.3:44997/ using document root <none> and password file <none>
I20260619 08:53:34.055917 30796 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:34.055999 30796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:34.057356 30796 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:34.057909 30811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:34.058085 30796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:53:34.058187 30796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "97a29f119d634665b64164dce5763aa9"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "88a64d88e27567cd2f858ccbb0d59242"
server_key_iv: "09a456128fc538d9afe9546de8ef7533"
server_key_version: "encryptionkey@0"
I20260619 08:53:34.058535 30796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:34.070580 30796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:34.072768 30796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:34.072924 30796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:34.073140 30796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:34.073429 30796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:34.073482 30796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:34.073514 30796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:34.073541 30796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:34.079380 30796 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:38145
I20260619 08:53:34.079425 30924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:38145 every 8 connection(s)
I20260619 08:53:34.079788 30796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:53:34 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859214, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:34.086676 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 30796
I20260619 08:53:34.089054 30658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.080477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:44839 (local address 127.15.208.62:38549)
0619 08:53:34.080524 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:34.080526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:34.080774 (+ 248us) server_negotiation.cc:407] Connection header received
0619 08:53:34.081299 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:34.081300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:34.081313 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:34.081378 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:34.082297 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.082789 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.083780 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.083975 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.084081 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.086083 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:34.086084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:34.086085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:34.086091 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:34.087631 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:34.088108 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:34.088109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:34.088109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:34.088169 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:34.088388 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:34.088388 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:34.088389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:34.088534 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:34.088604 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:34.088914 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 08:53:34.088992 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:53:34.089262 30927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.080476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:44839)
0619 08:53:34.080646 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.080679 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:34.081193 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:34.081446 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:34.081449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:34.082015 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:34.082196 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:34.082200 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.082885 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.082886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:34.083683 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:34.083685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.084088 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.084089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:34.084214 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.084631 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:34.084638 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:34.085990 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:34.087704 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:34.087711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:34.087714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:34.088033 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:34.088232 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:34.088232 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:34.088233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:34.088327 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:34.088607 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:34.088611 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:34.088769 (+ 158us) client_negotiation.cc:769] Sending connection context
0619 08:53:34.088881 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:53:34.089044 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260619 08:53:34.089869 30925 heartbeater.cc:347] Connected to a master server at 127.15.208.62:38549
I20260619 08:53:34.089970 30925 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:34.090159 30925 heartbeater.cc:511] Master 127.15.208.62:38549 requested a full tablet report, sending...
I20260619 08:53:34.090510 30467 ts_manager.cc:194] Registered new tserver with Master: 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145)
I20260619 08:53:34.090991 30467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:44839
I20260619 08:53:34.093797 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.087083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.15.208.62:38549)
0619 08:53:34.087210 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:34.087212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:34.087224 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:53:34.087283 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:34.087284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:34.087295 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:34.087366 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:34.088001 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.088680 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.089649 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.089948 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.090039 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.091487 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:34.091488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:34.091488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:34.091496 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:34.092654 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:34.093010 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:34.093011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:34.093012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:34.093081 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:34.093287 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:34.093287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:34.093288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:34.093396 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:34.093582 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:34.093620 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 08:53:34.093694 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260619 08:53:34.094583 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:53:34.097095 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.094923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34282 (local address 127.15.208.62:38549)
0619 08:53:34.094973 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:34.094975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:34.095058 (+ 83us) server_negotiation.cc:407] Connection header received
0619 08:53:34.095107 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:34.095107 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:34.095134 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:34.095193 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:53:34.095261 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.095698 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.096358 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.096545 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.096639 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.096796 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:53:34.096883 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:53:34.096942 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:34.097001 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 08:53:34.097038 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:53:34.097325 30467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:53:34.098965 30467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:53:34.101055 30589 tablet_service.cc:1467] Processing CreateTablet for tablet d789600271de466f8842b13ec84f1966 (DEFAULT_TABLE table=test-table [id=a15e43f4f486427bbcdfdd9c2c67f17e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:34.101260 30589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d789600271de466f8842b13ec84f1966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:34.102640 30547 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Bootstrap starting.
I20260619 08:53:34.102982 30547 tablet_bootstrap.cc:654] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:34.103871 30547 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: No bootstrap required, opened a new log
I20260619 08:53:34.103935 30547 ts_tablet_manager.cc:1397] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:53:34.104089 30547 raft_consensus.cc:348] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.104164 30547 raft_consensus.cc:374] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:34.104211 30547 raft_consensus.cc:729] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ebb143787a742a19e5b3adb46e39556, State: Initialized, Role: FOLLOWER
I20260619 08:53:34.104255 30547 consensus_queue.cc:260] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.104344 30547 ts_tablet_manager.cc:1428] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:53:34.104777 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.100692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:34759 (local address 127.0.0.1:43294)
0619 08:53:34.100785 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.100795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:34.100833 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:34.101284 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:34.101285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:34.101286 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:34.101354 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:34.101355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.102409 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.102410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:34.103566 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:34.103569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.104494 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.104495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:34.104572 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.104592 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 08:53:34.104650 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:53:34.104691 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 08:53:34.104804 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.100859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.15.208.2:34759)
0619 08:53:34.101054 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:34.101058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:34.101069 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:53:34.101108 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:34.101109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:34.101125 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:34.101211 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:34.101433 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.102305 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.103649 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.104424 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.104496 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.104551 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:34.104665 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 08:53:34.104700 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260619 08:53:34.105502 30727 tablet_service.cc:1467] Processing CreateTablet for tablet d789600271de466f8842b13ec84f1966 (DEFAULT_TABLE table=test-table [id=a15e43f4f486427bbcdfdd9c2c67f17e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:34.105849 30727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d789600271de466f8842b13ec84f1966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:34.106276 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.100692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:38145 (local address 127.0.0.1:53684)
0619 08:53:34.102678 (+ 1986us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.102689 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:34.102741 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:34.102939 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:34.102940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:34.102942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:34.102981 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:34.102983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.103995 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.103996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:34.105117 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:34.105120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.106053 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.106054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:34.106128 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.106131 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:53:34.106181 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:53:34.106218 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1971,"mutex_wait_us":49,"thread_start_us":54,"threads_started":1}
I20260619 08:53:34.106510 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.100873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.15.208.3:38145)
0619 08:53:34.101049 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:34.101056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:34.102799 (+ 1743us) server_negotiation.cc:407] Connection header received
0619 08:53:34.102846 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:34.102847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:34.102860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:34.102944 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:34.103035 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.103859 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.105212 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.105931 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.106015 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.106067 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:34.106396 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 08:53:34.106438 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260619 08:53:34.107069 30859 tablet_service.cc:1467] Processing CreateTablet for tablet d789600271de466f8842b13ec84f1966 (DEFAULT_TABLE table=test-table [id=a15e43f4f486427bbcdfdd9c2c67f17e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:34.107338 30859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d789600271de466f8842b13ec84f1966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:34.109072 30935 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Bootstrap starting.
I20260619 08:53:34.109696 30935 tablet_bootstrap.cc:654] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:34.110144 30936 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Bootstrap starting.
I20260619 08:53:34.110273 30935 log.cc:824] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:34.110795 30936 tablet_bootstrap.cc:654] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:34.111076 30935 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: No bootstrap required, opened a new log
I20260619 08:53:34.111157 30935 ts_tablet_manager.cc:1397] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:53:34.111371 30936 log.cc:824] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:34.112092 30936 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: No bootstrap required, opened a new log
I20260619 08:53:34.112159 30936 ts_tablet_manager.cc:1397] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:53:34.113184 30935 raft_consensus.cc:348] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.113368 30935 raft_consensus.cc:374] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:34.113402 30935 raft_consensus.cc:729] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e3611422a514e6bad8b5e873c01ae5d, State: Initialized, Role: FOLLOWER
I20260619 08:53:34.113529 30935 consensus_queue.cc:260] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.113809 30935 ts_tablet_manager.cc:1428] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:53:34.113819 30793 heartbeater.cc:503] Master 127.15.208.62:38549 was elected leader, sending a full tablet report...
I20260619 08:53:34.114141 30936 raft_consensus.cc:348] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.114284 30936 raft_consensus.cc:374] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:34.114311 30936 raft_consensus.cc:729] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a29f119d634665b64164dce5763aa9, State: Initialized, Role: FOLLOWER
I20260619 08:53:34.114382 30936 consensus_queue.cc:260] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.114647 30936 ts_tablet_manager.cc:1428] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 08:53:34.114661 30925 heartbeater.cc:503] Master 127.15.208.62:38549 was elected leader, sending a full tablet report...
W20260619 08:53:34.207288 30794 tablet.cc:2369] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:53:34.248816 30660 raft_consensus.cc:482] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:53:34.248939 30660 raft_consensus.cc:504] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.249248 30660 leader_election.cc:288] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145), 7e3611422a514e6bad8b5e873c01ae5d (127.15.208.2:34759)
I20260619 08:53:34.253213 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.249557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:59011 (local address 127.15.208.2:34759)
0619 08:53:34.249601 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:34.249603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:34.249610 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:53:34.249643 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:34.249645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:34.249657 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:34.249721 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:34.249914 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.250667 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.252167 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.252826 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.252901 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.252908 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:34.253103 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 08:53:34.253145 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:53:34.253243 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.249531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:44155 (local address 127.15.208.3:38145)
0619 08:53:34.249576 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:34.249578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:34.249642 (+ 64us) server_negotiation.cc:407] Connection header received
0619 08:53:34.249697 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:34.249698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:34.249708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:34.249759 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:34.249912 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.250659 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.252167 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.252826 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.252899 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.252908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:34.253103 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 08:53:34.253181 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:53:34.253257 30657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.249423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:34759 (local address 127.15.208.1:59011)
0619 08:53:34.249486 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.249497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:34.249549 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:34.249735 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:34.249736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:34.249754 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:34.249835 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:34.249836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.250745 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.250746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:34.252067 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:34.252071 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.252906 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.252908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:34.252994 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.253021 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 08:53:34.253085 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 08:53:34.253144 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":75}
I20260619 08:53:34.253263 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.249423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:38145 (local address 127.15.208.1:44155)
0619 08:53:34.249573 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.249582 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:34.249632 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:34.249773 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:34.249773 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:34.249774 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:34.249833 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:34.249835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.250745 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.250745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:34.252067 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:34.252071 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.252932 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.252933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:34.252994 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.253021 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 08:53:34.253083 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 08:53:34.253144 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":71,"thread_start_us":69,"threads_started":1}
I20260619 08:53:34.253670 30879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d789600271de466f8842b13ec84f1966" candidate_uuid: "4ebb143787a742a19e5b3adb46e39556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a29f119d634665b64164dce5763aa9" is_pre_election: true
I20260619 08:53:34.253747 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d789600271de466f8842b13ec84f1966" candidate_uuid: "4ebb143787a742a19e5b3adb46e39556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e3611422a514e6bad8b5e873c01ae5d" is_pre_election: true
I20260619 08:53:34.253852 30879 raft_consensus.cc:2454] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ebb143787a742a19e5b3adb46e39556 in term 0.
I20260619 08:53:34.253945 30747 raft_consensus.cc:2454] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ebb143787a742a19e5b3adb46e39556 in term 0.
I20260619 08:53:34.254096 30543 leader_election.cc:302] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ebb143787a742a19e5b3adb46e39556, 97a29f119d634665b64164dce5763aa9; no voters:
I20260619 08:53:34.254216 30660 raft_consensus.cc:2781] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:53:34.254288 30660 raft_consensus.cc:482] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:53:34.254305 30660 raft_consensus.cc:3037] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:34.255009 30660 raft_consensus.cc:504] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.255152 30660 leader_election.cc:288] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [CANDIDATE]: Term 1 election: Requested vote from peers 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145), 7e3611422a514e6bad8b5e873c01ae5d (127.15.208.2:34759)
I20260619 08:53:34.255287 30879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d789600271de466f8842b13ec84f1966" candidate_uuid: "4ebb143787a742a19e5b3adb46e39556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a29f119d634665b64164dce5763aa9"
I20260619 08:53:34.255316 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d789600271de466f8842b13ec84f1966" candidate_uuid: "4ebb143787a742a19e5b3adb46e39556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e3611422a514e6bad8b5e873c01ae5d"
I20260619 08:53:34.255398 30879 raft_consensus.cc:3037] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:34.255398 30747 raft_consensus.cc:3037] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:34.256125 30747 raft_consensus.cc:2454] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ebb143787a742a19e5b3adb46e39556 in term 1.
I20260619 08:53:34.256150 30879 raft_consensus.cc:2454] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ebb143787a742a19e5b3adb46e39556 in term 1.
I20260619 08:53:34.256295 30541 leader_election.cc:302] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ebb143787a742a19e5b3adb46e39556, 7e3611422a514e6bad8b5e873c01ae5d; no voters:
I20260619 08:53:34.256403 30660 raft_consensus.cc:2781] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:53:34.256562 30660 raft_consensus.cc:686] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 1 LEADER]: Becoming Leader. State: Replica: 4ebb143787a742a19e5b3adb46e39556, State: Running, Role: LEADER
I20260619 08:53:34.256659 30660 consensus_queue.cc:237] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.257434 30467 catalog_manager.cc:5495] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ebb143787a742a19e5b3adb46e39556 (127.15.208.1). New cstate: current_term: 1 leader_uuid: "4ebb143787a742a19e5b3adb46e39556" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } health_report { overall_health: HEALTHY } } }
I20260619 08:53:34.285843 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.283408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41536 (local address 127.15.208.1:43445)
0619 08:53:34.283481 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:34.283483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:34.283492 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:53:34.283544 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:34.283545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:34.283562 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:34.283639 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:53:34.283782 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.284271 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.285035 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:34.285251 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:34.285331 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:34.285502 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:53:34.285607 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:53:34.285687 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:34.285733 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 08:53:34.285754 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:53:34.287561 30879 raft_consensus.cc:1261] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 1 FOLLOWER]: Refusing update from remote peer 4ebb143787a742a19e5b3adb46e39556: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:53:34.287645 30747 raft_consensus.cc:1261] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Refusing update from remote peer 4ebb143787a742a19e5b3adb46e39556: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:53:34.287916 30660 consensus_queue.cc:1035] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:53:34.288048 30942 consensus_queue.cc:1035] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:53:34.291424 30946 mvcc.cc:205] Tried to move back new op lower bound from 7298495341719162880 to 7298495341596069888. Current Snapshot: MvccSnapshot[applied={T|T < 7298495341719162880}]
I20260619 08:53:34.292001 30947 mvcc.cc:205] Tried to move back new op lower bound from 7298495341719162880 to 7298495341596069888. Current Snapshot: MvccSnapshot[applied={T|T < 7298495341719162880}]
I20260619 08:53:34.293239 30467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:53:34.293318 30467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:53:34.294032 30467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Sending DeleteTablet for 3 replicas of tablet d789600271de466f8842b13ec84f1966
I20260619 08:53:34.294329 30589 tablet_service.cc:1514] Processing DeleteTablet for tablet d789600271de466f8842b13ec84f1966 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:34 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:41526
I20260619 08:53:34.294363 30727 tablet_service.cc:1514] Processing DeleteTablet for tablet d789600271de466f8842b13ec84f1966 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:34 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:43294
I20260619 08:53:34.294420 30664 tablet_replica.cc:330] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: stopping tablet replica
I20260619 08:53:34.294485 30664 raft_consensus.cc:2229] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:53:34.294462 30859 tablet_service.cc:1514] Processing DeleteTablet for tablet d789600271de466f8842b13ec84f1966 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:34 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53684
I20260619 08:53:34.294579 30664 raft_consensus.cc:2258] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:34.294735 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 30505
I20260619 08:53:34.294813 30954 tablet_replica.cc:330] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: stopping tablet replica
I20260619 08:53:34.294870 30664 ts_tablet_manager.cc:1905] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:53:34.295383 30954 raft_consensus.cc:2229] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:53:34.295492 30954 raft_consensus.cc:2258] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:34.295794 30955 tablet_replica.cc:330] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: stopping tablet replica
I20260619 08:53:34.295858 30955 raft_consensus.cc:2229] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:53:34.295919 30955 raft_consensus.cc:2258] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:34.296277 30955 ts_tablet_manager.cc:1905] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:53:34.297737 30955 ts_tablet_manager.cc:1918] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:53:34.297793 30955 log.cc:1196] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/d789600271de466f8842b13ec84f1966
I20260619 08:53:34.298051 30955 ts_tablet_manager.cc:1939] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Deleting consensus metadata
I20260619 08:53:34.298450 30453 catalog_manager.cc:4841] TS 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145): tablet d789600271de466f8842b13ec84f1966 (table test-table [id=a15e43f4f486427bbcdfdd9c2c67f17e]) successfully deleted
W20260619 08:53:34.300465 30454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:43445 (error 108)
I20260619 08:53:34.300906 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 30661
I20260619 08:53:34.301043 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.300895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43445 (local address 127.0.0.1:41540)
0619 08:53:34.300960 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.301003 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:43445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 08:53:34.302438 30454 catalog_manager.cc:4584] TS 4ebb143787a742a19e5b3adb46e39556 (127.15.208.1:43445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d789600271de466f8842b13ec84f1966: Network error: Client connection negotiation failed: client connection to 127.15.208.1:43445: connect: Connection refused (error 111)
I20260619 08:53:34.305944 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 30796
I20260619 08:53:34.305977 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.305868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:34759 (local address 127.0.0.1:43296)
0619 08:53:34.305924 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.305953 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.2:34759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 08:53:34.310348 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 30434
I20260619 08:53:34.315660 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:38549
--webserver_interface=127.15.208.62
--webserver_port=38917
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:38549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:34.400179 30957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:34.400425 30957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:34.400462 30957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:34.401716 30957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:53:34.401784 30957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:34.401810 30957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:34.401839 30957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:53:34.401870 30957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:53:34.403389 30957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:38549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:38549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=38917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:34.403671 30957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:34.403898 30957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:34.408918 30962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:34.408941 30965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:34.408999 30963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:34.409186 30957 server_base.cc:1034] running on GCE node
I20260619 08:53:34.409440 30957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:34.409766 30957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:34.410923 30957 hybrid_clock.cc:648] HybridClock initialized: now 1781859214410911 us; error 28 us; skew 500 ppm
Jun 19 08:53:34 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859214, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:34.413478 30957 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:53:34.413936 30957 webserver.cc:460] Webserver started at http://127.15.208.62:38917/ using document root <none> and password file <none>
I20260619 08:53:34.414181 30957 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:34.414250 30957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:34.415550 30957 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:53:34.416200 30972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:34.416368 30957 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:53:34.416466 30957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "8e7ad409e2334281a779e7597c57fd37"
format_stamp: "Formatted at 2026-06-19 08:53:32 on dist-test-slave-p0h9"
server_key: "8b2d85e1c3369a0b806f5a952b2cf9f6"
server_key_iv: "6b41e6a55b9908bfc5a17b37f51bfb12"
server_key_version: "encryptionkey@0"
I20260619 08:53:34.416868 30957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:34.433161 30957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:34.435005 30957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:34.435168 30957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:34.439291 30957 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:38549
I20260619 08:53:34.439327 31024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:38549 every 8 connection(s)
I20260619 08:53:34.439713 30957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:53:34.440272 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 30957
I20260619 08:53:34.440627 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:43445
--local_ip_for_outbound_sockets=127.15.208.1
--tserver_master_addrs=127.15.208.62:38549
--webserver_port=32925
--webserver_interface=127.15.208.1
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:53:34.442065 31025 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:53:34.442721 31025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Bootstrap starting.
I20260619 08:53:34.454241 31025 log.cc:824] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:34.457120 31025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:53:34.457448 31025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37: Bootstrap complete.
I20260619 08:53:34.460520 31025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:34.460915 31025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e7ad409e2334281a779e7597c57fd37, State: Initialized, Role: FOLLOWER
I20260619 08:53:34.461099 31025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:34.461174 31025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:53:34.461218 31025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:53:34.461262 31025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 2 FOLLOWER]: Advancing to term 3
I20260619 08:53:34.462508 31025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:34.462610 31025 leader_election.cc:302] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e7ad409e2334281a779e7597c57fd37; no voters:
I20260619 08:53:34.462757 31025 leader_election.cc:288] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 08:53:34.462826 31029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 08:53:34.462991 31029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [term 3 LEADER]: Becoming Leader. State: Replica: 8e7ad409e2334281a779e7597c57fd37, State: Running, Role: LEADER
I20260619 08:53:34.463016 31025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:53:34.463086 31029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } }
I20260619 08:53:34.463371 31030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8e7ad409e2334281a779e7597c57fd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } } }
I20260619 08:53:34.463444 31030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:34.463694 31031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e7ad409e2334281a779e7597c57fd37. Latest consensus state: current_term: 3 leader_uuid: "8e7ad409e2334281a779e7597c57fd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7ad409e2334281a779e7597c57fd37" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38549 } } }
I20260619 08:53:34.463828 31031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e7ad409e2334281a779e7597c57fd37 [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:34.463936 31035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:53:34.464694 31035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:53:34.464829 31035 catalog_manager.cc:1259] Loaded cluster ID: fa97d6ac7c8b46c6a72773e039676602
I20260619 08:53:34.464871 31035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 08:53:34.465235 31035 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1a0060e9ad google::LogMessage::Fail() at ??:0
@ 0x7f1a0061188a google::LogMessage::SendToLog() at ??:0
@ 0x7f1a0060e4de google::LogMessage::Flush() at ??:0
@ 0x7f1a0060fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1a0852f0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f1a085388b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1a024dc0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1a024d1954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1a033356db start_thread at ??:0
@ 0x7f19ff10c61f clone at ??:0
W20260619 08:53:34.545009 31027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:34.545195 31027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:34.545217 31027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:34.546389 31027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:34.546432 31027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:34.546458 31027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:53:34.547946 31027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:43445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=32925
--tserver_master_addrs=127.15.208.62:38549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:34.548171 31027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:34.548415 31027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:34.553297 31062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:34.553277 31061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:34.553268 31064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:34.553623 31027 server_base.cc:1034] running on GCE node
I20260619 08:53:34.553818 31027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:34.554073 31027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:34.555241 31027 hybrid_clock.cc:648] HybridClock initialized: now 1781859214555220 us; error 41 us; skew 500 ppm
Jun 19 08:53:34 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859214, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:34.557804 31027 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:53:34.558269 31027 webserver.cc:460] Webserver started at http://127.15.208.1:32925/ using document root <none> and password file <none>
I20260619 08:53:34.558503 31027 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:34.558570 31027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:34.559875 31027 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:34.560660 31071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:34.560847 31027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:34.560949 31027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "4ebb143787a742a19e5b3adb46e39556"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "670efa30942381b26024f83cc28c8516"
server_key_iv: "41014ea0dfd49a48f32bc8c908cd157f"
server_key_version: "encryptionkey@0"
I20260619 08:53:34.561299 31027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:34.580032 31027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:34.582314 31027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:34.582492 31027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:34.582733 31027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:34.583212 31078 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:53:34.585114 31027 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:53:34.585189 31027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:53:34.585238 31027 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:53:34.585855 31027 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:53:34.585912 31027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:53:34.585983 31078 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Bootstrap starting.
I20260619 08:53:34.587883 31078 log.cc:824] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:34.590773 31078 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:53:34.591081 31078 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Bootstrap complete.
I20260619 08:53:34.591238 31078 ts_tablet_manager.cc:1397] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 08:53:34.593153 31027 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:43445
I20260619 08:53:34.593183 31186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:43445 every 8 connection(s)
I20260619 08:53:34.593525 31027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:53:34.593240 31078 raft_consensus.cc:348] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.593628 31078 raft_consensus.cc:729] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ebb143787a742a19e5b3adb46e39556, State: Initialized, Role: FOLLOWER
I20260619 08:53:34.594168 31078 consensus_queue.cc:260] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.594473 31078 ts_tablet_manager.cc:1428] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:53:34.596802 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 31027
I20260619 08:53:34.597062 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:34759
--local_ip_for_outbound_sockets=127.15.208.2
--tserver_master_addrs=127.15.208.62:38549
--webserver_port=38229
--webserver_interface=127.15.208.2
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:34.644956 31189 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:53:34.594164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:36171)
0619 08:53:34.594332 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.594373 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:34.594527 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:34.644706 (+ 50179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
W20260619 08:53:34.645262 31187 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:38549 (0 consecutive failures): Network error: Failed to ping master at 127.15.208.62:38549: Client connection negotiation failed: client connection to 127.15.208.62:38549: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:53:34.645637 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.645509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:37439)
0619 08:53:34.645565 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.645611 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 08:53:34.645944 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.645856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:43273)
0619 08:53:34.645902 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.645927 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:53:34.646073 31187 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:53:34.680043 31191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:34.680233 31191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:34.680256 31191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:34.681450 31191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:34.681494 31191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:34.681519 31191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:53:34.682976 31191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:34759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=38229
--tserver_master_addrs=127.15.208.62:38549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:34.683238 31191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:34.683470 31191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:34.688303 31197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:34.688294 31196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:34.688297 31199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:34.688549 31191 server_base.cc:1034] running on GCE node
I20260619 08:53:34.688805 31191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:34.689065 31191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:34.690204 31191 hybrid_clock.cc:648] HybridClock initialized: now 1781859214690159 us; error 64 us; skew 500 ppm
Jun 19 08:53:34 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859214, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:34.692657 31191 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:53:34.693143 31191 webserver.cc:460] Webserver started at http://127.15.208.2:38229/ using document root <none> and password file <none>
I20260619 08:53:34.693373 31191 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:34.693418 31191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:34.694794 31191 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:34.695451 31206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:34.695631 31191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:53:34.695719 31191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "7e3611422a514e6bad8b5e873c01ae5d"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "407eee8b7588541b049d49b87ce9121c"
server_key_iv: "ad6035933b58045e55a2db24d85d4c3c"
server_key_version: "encryptionkey@0"
I20260619 08:53:34.696048 31191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:34.712437 31191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:34.714325 31191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:34.714493 31191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:34.714748 31191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:34.715189 31213 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:53:34.716570 31191 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:53:34.716632 31191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:53:34.716682 31191 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:53:34.717326 31191 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:53:34.717396 31191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:53:34.717417 31213 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Bootstrap starting.
I20260619 08:53:34.718976 31213 log.cc:824] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:34.721431 31213 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:53:34.721726 31213 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Bootstrap complete.
I20260619 08:53:34.721890 31213 ts_tablet_manager.cc:1397] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 08:53:34.723362 31213 raft_consensus.cc:348] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.723608 31213 raft_consensus.cc:729] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e3611422a514e6bad8b5e873c01ae5d, State: Initialized, Role: FOLLOWER
I20260619 08:53:34.723790 31213 consensus_queue.cc:260] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:34.724363 31213 ts_tablet_manager.cc:1428] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:53:34.725102 31191 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:34759
I20260619 08:53:34.725188 31322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:34759 every 8 connection(s)
I20260619 08:53:34.725462 31191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:53:34.726362 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.726064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:51579)
0619 08:53:34.726193 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.726255 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
W20260619 08:53:34.726548 31323 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:38549 (0 consecutive failures): Network error: Failed to ping master at 127.15.208.62:38549: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
I20260619 08:53:34.726826 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.726745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:38761)
0619 08:53:34.726786 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.726810 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 08:53:34.727041 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.726994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:57183)
0619 08:53:34.727012 (+ 18us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.727029 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260619 08:53:34.727133 31323 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:53:34.733045 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 31191
I20260619 08:53:34.733336 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:38145
--local_ip_for_outbound_sockets=127.15.208.3
--tserver_master_addrs=127.15.208.62:38549
--webserver_port=44997
--webserver_interface=127.15.208.3
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:34.818224 31326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:34.818432 31326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:34.818455 31326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:34.819527 31326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:34.819562 31326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:34.819587 31326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:53:34.820948 31326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:38145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=44997
--tserver_master_addrs=127.15.208.62:38549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:34.821141 31326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:34.821323 31326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:34.826179 31334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:34.826258 31326 server_base.cc:1034] running on GCE node
W20260619 08:53:34.826151 31331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:34.826154 31332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:34.826606 31326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:34.826848 31326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:34.828007 31326 hybrid_clock.cc:648] HybridClock initialized: now 1781859214827977 us; error 62 us; skew 500 ppm
Jun 19 08:53:34 dist-test-slave-p0h9 krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859214, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:34.834972 31326 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:53:34.835461 31326 webserver.cc:460] Webserver started at http://127.15.208.3:44997/ using document root <none> and password file <none>
I20260619 08:53:34.835695 31326 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:34.835767 31326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:34.837149 31326 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:34.837750 31341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:34.838066 31326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:34.838171 31326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "97a29f119d634665b64164dce5763aa9"
format_stamp: "Formatted at 2026-06-19 08:53:33 on dist-test-slave-p0h9"
server_key: "88a64d88e27567cd2f858ccbb0d59242"
server_key_iv: "09a456128fc538d9afe9546de8ef7533"
server_key_version: "encryptionkey@0"
I20260619 08:53:34.838513 31326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:34.879830 31326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:34.881843 31326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:34.882019 31326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:34.882282 31326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:34.882623 31326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:34.882679 31326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:34.882722 31326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:34.882751 31326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:34.888720 31326 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:38145
I20260619 08:53:34.888808 31454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:38145 every 8 connection(s)
I20260619 08:53:34.889114 31326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:53:34.890156 31457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.889763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:57895)
0619 08:53:34.889914 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.889988 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260619 08:53:34.890573 31455 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:38549 (0 consecutive failures): Network error: Failed to ping master at 127.15.208.62:38549: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
I20260619 08:53:34.890857 31457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.890754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:59265)
0619 08:53:34.890802 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.890828 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 08:53:34.891124 31457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:34.891039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:54877)
0619 08:53:34.891081 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:34.891103 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 08:53:34.891232 31455 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:53:34.898809 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 31326
I20260619 08:53:35.646634 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:35.646397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:40343)
0619 08:53:35.646539 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:35.646572 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":60,"threads_started":1}
I20260619 08:53:35.727651 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:35.727441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:40297)
0619 08:53:35.727572 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:35.727609 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260619 08:53:35.891785 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:35.891547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:58451)
0619 08:53:35.891691 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:35.891726 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260619 08:53:36.033710 31462 raft_consensus.cc:482] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:53:36.033824 31462 raft_consensus.cc:504] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:36.034102 31462 leader_election.cc:288] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145), 4ebb143787a742a19e5b3adb46e39556 (127.15.208.1:43445)
Jun 19 08:53:36 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859214, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.3@KRBTEST.COM
Jun 19 08:53:36 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859214, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:53:36.045643 31465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:36.034375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:45147 (local address 127.15.208.3:38145)
0619 08:53:36.034537 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:36.034548 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:36.034582 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:53:36.035157 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:36.035160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:36.035198 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:36.035299 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:36.036038 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:36.037240 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:36.038239 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:36.038849 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:36.038963 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:36.041729 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:36.041733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:36.041733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:36.041745 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:36.043864 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:36.044283 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:36.044286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:36.044287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:36.044397 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:36.044646 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:36.044647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:36.044647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:36.045036 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:36.045113 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:36.045395 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 08:53:36.045536 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":61,"threads_started":1}
I20260619 08:53:36.045768 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:36.034379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:36195 (local address 127.15.208.1:43445)
0619 08:53:36.034539 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:36.034544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:36.034560 (+ 16us) server_negotiation.cc:407] Connection header received
0619 08:53:36.034710 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:36.034712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:36.034735 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:36.034840 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:36.035597 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:36.036927 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:36.038239 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:36.038849 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:36.038962 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:36.042223 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:36.042226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:36.042227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:36.042236 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:36.044341 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:36.044688 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:36.044690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:36.044690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:36.044826 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:36.045015 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:36.045016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:36.045016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:36.045220 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:36.045313 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:36.045494 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 08:53:36.045661 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
I20260619 08:53:36.045646 31463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:36.034254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:38145 (local address 127.15.208.2:45147)
0619 08:53:36.034405 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:36.034414 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:36.034553 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:36.035309 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:36.035311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:36.035854 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:36.035903 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:36.035905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:36.037321 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:36.037322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:36.038145 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:36.038147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:36.038943 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:36.038944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:36.039121 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:36.039744 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:36.039748 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:36.041596 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:36.043964 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:36.043971 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:36.043974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:36.044195 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:36.044467 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:36.044468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:36.044468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:36.044582 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:36.045090 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:36.045096 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:36.045230 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 08:53:36.045331 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 08:53:36.045521 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":55,"thread_start_us":63,"threads_started":1}
I20260619 08:53:36.045760 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:36.034254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43445 (local address 127.15.208.2:36195)
0619 08:53:36.034292 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:36.034321 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:36.034550 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:36.034885 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:36.034888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:36.035170 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:36.035450 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:36.035457 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:36.037014 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:36.037015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:36.038145 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:36.038147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:36.038945 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:36.038945 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:36.039121 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:36.039722 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:36.039733 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:36.042125 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:36.044418 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:36.044419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:36.044419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:36.044600 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:36.044890 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:36.044891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:36.044891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:36.044950 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:36.045297 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:36.045298 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:36.045344 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 08:53:36.045432 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:53:36.045609 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":95}
I20260619 08:53:36.046478 31141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d789600271de466f8842b13ec84f1966" candidate_uuid: "7e3611422a514e6bad8b5e873c01ae5d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4ebb143787a742a19e5b3adb46e39556" is_pre_election: true
I20260619 08:53:36.046648 31141 raft_consensus.cc:2454] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e3611422a514e6bad8b5e873c01ae5d in term 1.
W20260619 08:53:36.046838 31209 leader_election.cc:341] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145): Not found: Tablet not found: d789600271de466f8842b13ec84f1966
I20260619 08:53:36.047041 31210 leader_election.cc:302] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ebb143787a742a19e5b3adb46e39556, 7e3611422a514e6bad8b5e873c01ae5d; no voters: 97a29f119d634665b64164dce5763aa9
I20260619 08:53:36.047183 31462 raft_consensus.cc:2781] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 08:53:36.047266 31462 raft_consensus.cc:482] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:53:36.047302 31462 raft_consensus.cc:3037] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:53:36.048429 31462 raft_consensus.cc:504] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:36.048588 31462 leader_election.cc:288] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [CANDIDATE]: Term 2 election: Requested vote from peers 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145), 4ebb143787a742a19e5b3adb46e39556 (127.15.208.1:43445)
I20260619 08:53:36.048780 31141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d789600271de466f8842b13ec84f1966" candidate_uuid: "7e3611422a514e6bad8b5e873c01ae5d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4ebb143787a742a19e5b3adb46e39556"
I20260619 08:53:36.048870 31141 raft_consensus.cc:3037] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 1 FOLLOWER]: Advancing to term 2
W20260619 08:53:36.048952 31209 leader_election.cc:341] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145): Not found: Tablet not found: d789600271de466f8842b13ec84f1966
I20260619 08:53:36.050045 31141 raft_consensus.cc:2454] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e3611422a514e6bad8b5e873c01ae5d in term 2.
I20260619 08:53:36.050226 31210 leader_election.cc:302] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ebb143787a742a19e5b3adb46e39556, 7e3611422a514e6bad8b5e873c01ae5d; no voters: 97a29f119d634665b64164dce5763aa9
I20260619 08:53:36.050318 31462 raft_consensus.cc:2781] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:53:36.050436 31462 raft_consensus.cc:686] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [term 2 LEADER]: Becoming Leader. State: Replica: 7e3611422a514e6bad8b5e873c01ae5d, State: Running, Role: LEADER
I20260619 08:53:36.050539 31462 consensus_queue.cc:237] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:36.050746 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:36.050639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:58437)
0619 08:53:36.050678 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:36.050702 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 08:53:36.511874 31141 raft_consensus.cc:1261] T d789600271de466f8842b13ec84f1966 P 4ebb143787a742a19e5b3adb46e39556 [term 2 FOLLOWER]: Refusing update from remote peer 7e3611422a514e6bad8b5e873c01ae5d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260619 08:53:36.512305 31462 consensus_queue.cc:1035] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 08:53:36.513326 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:36.513145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:50053)
0619 08:53:36.513223 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:36.513280 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62,"thread_start_us":46,"threads_started":1}
I20260619 08:53:36.514137 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:36.514017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:60977)
0619 08:53:36.514079 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:36.514104 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 08:53:36.514258 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:36.514161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:40511)
0619 08:53:36.514200 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:36.514225 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 08:53:36.514259 31209 consensus_peers.cc:487] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d -> Peer 97a29f119d634665b64164dce5763aa9 (127.15.208.3:38145): Couldn't send request to peer 97a29f119d634665b64164dce5763aa9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d789600271de466f8842b13ec84f1966. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:53:36.892433 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:36.892244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:40099)
0619 08:53:36.892369 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:36.892398 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260619 08:53:36.999400 31474 ts_tablet_manager.cc:927] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Initiating tablet copy from peer 7e3611422a514e6bad8b5e873c01ae5d (127.15.208.2:34759)
I20260619 08:53:36.999805 31474 tablet_copy_client.cc:316] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.208.2:34759
Jun 19 08:53:37 dist-test-slave-p0h9 krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859214, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.2@KRBTEST.COM
I20260619 08:53:37.007768 31476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:37.000196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:55367 (local address 127.15.208.2:34759)
0619 08:53:37.000326 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:37.000331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:37.000340 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:53:37.000363 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:37.000365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:37.000377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:37.000440 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:37.000935 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:37.001597 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:37.002334 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:37.002614 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:37.002666 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:37.004717 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:37.004748 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:37.004748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:37.004756 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:37.006470 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:37.006856 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:37.006860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:37.006860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:37.006922 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:37.007129 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:37.007130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:37.007131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:37.007284 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:37.007379 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:37.007563 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 08:53:37.007657 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260619 08:53:37.007771 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:37.000073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:34759 (local address 127.15.208.3:55367)
0619 08:53:37.000178 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:37.000199 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:37.000278 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:37.000456 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:37.000460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:37.000793 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:37.000864 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:37.000867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:37.001667 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:37.001668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:37.002261 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:37.002263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:37.002676 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:37.002677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:37.002728 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:37.003226 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:37.003230 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:37.004612 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:37.006556 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:37.006563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:37.006565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:37.006767 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:37.006992 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:37.006993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:37.006994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:37.007064 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:37.007359 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:37.007366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:37.007420 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 08:53:37.007533 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 08:53:37.007656 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 08:53:37.008273 31296 tablet_copy_service.cc:140] P 7e3611422a514e6bad8b5e873c01ae5d: Received BeginTabletCopySession request for tablet d789600271de466f8842b13ec84f1966 from peer 97a29f119d634665b64164dce5763aa9 ({username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:55367)
I20260619 08:53:37.008342 31296 tablet_copy_service.cc:161] P 7e3611422a514e6bad8b5e873c01ae5d: Beginning new tablet copy session on tablet d789600271de466f8842b13ec84f1966 from peer 97a29f119d634665b64164dce5763aa9 at {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:55367: session id = 97a29f119d634665b64164dce5763aa9-d789600271de466f8842b13ec84f1966
I20260619 08:53:37.008896 31296 tablet_copy_source_session.cc:205] T d789600271de466f8842b13ec84f1966 P 7e3611422a514e6bad8b5e873c01ae5d: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 08:53:37.009850 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d789600271de466f8842b13ec84f1966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:37.012622 31474 tablet_copy_client.cc:791] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: tablet copy: Starting download of 0 data blocks...
I20260619 08:53:37.012806 31474 tablet_copy_client.cc:655] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: tablet copy: Starting download of 1 WAL segments...
I20260619 08:53:37.013655 31474 tablet_copy_client.cc:531] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 08:53:37.014424 31474 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Bootstrap starting.
I20260619 08:53:37.015733 31474 log.cc:824] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:37.018200 31474 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:53:37.018424 31474 tablet_bootstrap.cc:492] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Bootstrap complete.
I20260619 08:53:37.018565 31474 ts_tablet_manager.cc:1397] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260619 08:53:37.019446 31474 raft_consensus.cc:348] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:37.019603 31474 raft_consensus.cc:729] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a29f119d634665b64164dce5763aa9, State: Initialized, Role: FOLLOWER
I20260619 08:53:37.019727 31474 consensus_queue.cc:260] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a29f119d634665b64164dce5763aa9" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38145 } } peers { permanent_uuid: "7e3611422a514e6bad8b5e873c01ae5d" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34759 } } peers { permanent_uuid: "4ebb143787a742a19e5b3adb46e39556" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43445 } }
I20260619 08:53:37.020004 31474 ts_tablet_manager.cc:1428] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9: Time spent starting tablet: real 0.001s user 0.000s sys 0.003s
I20260619 08:53:37.020273 31296 tablet_copy_service.cc:342] P 7e3611422a514e6bad8b5e873c01ae5d: Request end of tablet copy session 97a29f119d634665b64164dce5763aa9-d789600271de466f8842b13ec84f1966 received from {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:55367
I20260619 08:53:37.020331 31296 tablet_copy_service.cc:433] P 7e3611422a514e6bad8b5e873c01ae5d: ending tablet copy session 97a29f119d634665b64164dce5763aa9-d789600271de466f8842b13ec84f1966 on tablet d789600271de466f8842b13ec84f1966 with peer 97a29f119d634665b64164dce5763aa9
I20260619 08:53:37.020305 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:37.020204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:37153)
0619 08:53:37.020249 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:37.020281 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 08:53:37.020639 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:37.020546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:50201)
0619 08:53:37.020577 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:37.020611 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 08:53:37.514819 31479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:37.514620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:34941)
0619 08:53:37.514733 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:37.514773 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
I20260619 08:53:37.514853 31480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:37.514642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:34437)
0619 08:53:37.514767 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:37.514810 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260619 08:53:37.550385 31479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:37.550256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:59675)
0619 08:53:37.550313 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:37.550349 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:53:37.550366 31409 raft_consensus.cc:1203] T d789600271de466f8842b13ec84f1966 P 97a29f119d634665b64164dce5763aa9 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260619 08:53:37.550982 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:37.550788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:36327)
0619 08:53:37.550904 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:37.550942 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":50,"threads_started":1}
I20260619 08:53:38.515620 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:38.515374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:38197)
0619 08:53:38.515514 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:38.515571 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260619 08:53:38.550946 31485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:38.550744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:55685)
0619 08:53:38.550871 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:38.550901 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260619 08:53:38.551532 31486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:38.551343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:58443)
0619 08:53:38.551456 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:38.551495 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":49,"threads_started":1}
I20260619 08:53:39.516315 31488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:39.516085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:55503)
0619 08:53:39.516231 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:39.516266 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260619 08:53:39.551628 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:39.551413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:59699)
0619 08:53:39.551537 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:39.551571 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260619 08:53:39.552095 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:39.551939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:45377)
0619 08:53:39.552026 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:39.552060 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":43,"threads_started":1}
I20260619 08:53:40.517140 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:40.516885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:57519)
0619 08:53:40.517052 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:40.517093 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":64,"threads_started":1}
I20260619 08:53:40.552388 31492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:40.552137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:56135)
0619 08:53:40.552284 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:40.552329 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260619 08:53:40.552834 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:40.552576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:54501)
0619 08:53:40.552698 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:40.552789 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260619 08:53:41.517946 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:41.517712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:57369)
0619 08:53:41.517859 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:41.517899 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260619 08:53:41.553229 31496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:41.552972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:36787)
0619 08:53:41.553143 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:41.553183 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
I20260619 08:53:41.553488 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:41.553215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:42799)
0619 08:53:41.553415 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:41.553456 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":94,"threads_started":1}
I20260619 08:53:42.518718 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:42.518450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:44103)
0619 08:53:42.518621 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:42.518666 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":74,"threads_started":1}
I20260619 08:53:42.553916 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:42.553675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:41933)
0619 08:53:42.553835 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:42.553871 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":67,"threads_started":1}
I20260619 08:53:42.554236 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:42.553905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:49479)
0619 08:53:42.554137 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:42.554178 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":73,"threads_started":1}
I20260619 08:53:43.519460 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:43.519216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:47601)
0619 08:53:43.519344 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:43.519379 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260619 08:53:43.554741 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:43.554487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:52201)
0619 08:53:43.554639 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:43.554680 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260619 08:53:43.554944 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:43.554696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:42495)
0619 08:53:43.554853 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:43.554895 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260619 08:53:44.520192 31505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:44.519943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:44313)
0619 08:53:44.520102 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:44.520143 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":62,"threads_started":1}
I20260619 08:53:44.555594 31506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:44.555331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:35573)
0619 08:53:44.555509 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:44.555547 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":120,"threads_started":1}
I20260619 08:53:44.555703 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:44.555458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:51115)
0619 08:53:44.555601 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:44.555633 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260619 08:53:45.520987 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:45.520713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:52719)
0619 08:53:45.520907 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:45.520938 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":135,"threads_started":1}
I20260619 08:53:45.556365 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:45.556112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:46105)
0619 08:53:45.556261 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:45.556304 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260619 08:53:45.556584 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:45.556258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:43455)
0619 08:53:45.556411 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:45.556453 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260619 08:53:46.521744 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:46.521519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:33453)
0619 08:53:46.521661 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:46.521697 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260619 08:53:46.557168 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:46.556933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:47415)
0619 08:53:46.557083 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:46.557122 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260619 08:53:46.557201 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:46.557017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:32947)
0619 08:53:46.557122 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:46.557160 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":58,"threads_started":1}
I20260619 08:53:47.522563 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:47.522288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:40315)
0619 08:53:47.522463 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:47.522514 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":62,"threads_started":1}
I20260619 08:53:47.558076 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:47.557810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:35505)
0619 08:53:47.557978 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:47.558014 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":78,"threads_started":1}
I20260619 08:53:47.558071 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:47.557810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:34357)
0619 08:53:47.557963 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:47.558024 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260619 08:53:48.523336 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:48.523086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:47569)
0619 08:53:48.523243 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:48.523279 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260619 08:53:48.558832 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:48.558575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:36101)
0619 08:53:48.558728 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:48.558773 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260619 08:53:48.558831 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:48.558580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:50569)
0619 08:53:48.558713 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:48.558766 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260619 08:53:49.524081 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:49.523821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.1:56763)
0619 08:53:49.523994 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:49.524034 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
I20260619 08:53:49.559530 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:49.559312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.2:60915)
0619 08:53:49.559442 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:49.559475 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260619 08:53:49.559738 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:49.559479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38549 (local address 127.15.208.3:35307)
0619 08:53:49.559614 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:49.559684 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260619 08:53:49.901351 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 31027
I20260619 08:53:49.906258 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 31191
I20260619 08:53:49.910912 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 31326
2026-06-19T08:53:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18306 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:53:49 dist-test-slave-p0h9 krb5kdc[31527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:53:49 dist-test-slave-p0h9 krb5kdc[31527](info): set up 2 sockets
Jun 19 08:53:49 dist-test-slave-p0h9 krb5kdc[31527](info): commencing operation
krb5kdc: starting...
W20260619 08:53:50.953291 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:53:51 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:53:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:53:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:53:51.071266 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39305
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:39305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:51.151248 31540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:51.151465 31540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:51.151494 31540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:51.152667 31540 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 08:53:51.152710 31540 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 08:53:51.152783 31540 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 08:53:51.152810 31540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:53:51.152835 31540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:51.152864 31540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:51.152920 31540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:53:51.152942 31540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:53:51.154454 31540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:39305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:51.154731 31540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:51.154990 31540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:51.160238 31545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:51.160282 31540 server_base.cc:1034] running on GCE node
W20260619 08:53:51.160296 31546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:51.160243 31548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:51.160933 31540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:51.161237 31540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:51.162405 31540 hybrid_clock.cc:648] HybridClock initialized: now 1781859231162398 us; error 37 us; skew 500 ppm
Jun 19 08:53:51 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859231, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:51.164978 31540 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:53:51.165426 31540 webserver.cc:460] Webserver started at http://127.15.208.62:32787/ using document root <none> and password file <none>
I20260619 08:53:51.165648 31540 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:51.165691 31540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:51.165778 31540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:51.166777 31540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "3521573a48134e558acf607515f56deb"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "6d63c4561cfcc4fdd1c7ce6bcce252a9"
server_key_iv: "08fdb93dfa7fffa7f9a6f16c98dc82cf"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.167076 31540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "3521573a48134e558acf607515f56deb"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "6d63c4561cfcc4fdd1c7ce6bcce252a9"
server_key_iv: "08fdb93dfa7fffa7f9a6f16c98dc82cf"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.168294 31540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:53:51.169036 31555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.169214 31540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:53:51.169317 31540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "3521573a48134e558acf607515f56deb"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "6d63c4561cfcc4fdd1c7ce6bcce252a9"
server_key_iv: "08fdb93dfa7fffa7f9a6f16c98dc82cf"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.169389 31540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:51.192277 31540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:51.194686 31540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:51.194849 31540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:51.199082 31540 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:39305
I20260619 08:53:51.199116 31607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:39305 every 8 connection(s)
I20260619 08:53:51.199467 31540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:53:51.200369 31608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:51.202883 31608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Bootstrap starting.
I20260619 08:53:51.203538 31608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:51.203951 31608 log.cc:824] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:51.204666 31608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: No bootstrap required, opened a new log
I20260619 08:53:51.206013 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 31540
I20260619 08:53:51.206110 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:53:51.206302 16192 external_mini_cluster.cc:1428] Setting key 4749ee7c36d6eed7fbede441e6c87883
I20260619 08:53:51.206418 31608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:53:51.206579 31608 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:51.206629 31608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3521573a48134e558acf607515f56deb, State: Initialized, Role: FOLLOWER
I20260619 08:53:51.206804 31608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:53:51.206902 31608 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:53:51.206972 31608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:53:51.207041 31608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:51.208077 31608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:53:51.208190 31608 leader_election.cc:302] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3521573a48134e558acf607515f56deb; no voters:
I20260619 08:53:51.208338 31608 leader_election.cc:288] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:53:51.208412 31613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:53:51.208585 31613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 1 LEADER]: Becoming Leader. State: Replica: 3521573a48134e558acf607515f56deb, State: Running, Role: LEADER
I20260619 08:53:51.208588 31608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:53:51.208690 31613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:53:51.209139 31615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3521573a48134e558acf607515f56deb. Latest consensus state: current_term: 1 leader_uuid: "3521573a48134e558acf607515f56deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } } }
I20260619 08:53:51.209370 31615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:51.209303 31614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3521573a48134e558acf607515f56deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } } }
I20260619 08:53:51.209616 31614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: This master's current role is: LEADER
I20260619 08:53:51.209730 31618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:53:51.210208 31618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:53:51.211962 31618 catalog_manager.cc:1347] Generated new cluster ID: 6b2413d248de4759abf85340654d2686
I20260619 08:53:51.212014 31618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:53:51 dist-test-slave-p0h9 krb5kdc[31527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:51.217825 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.206821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52696 (local address 127.15.208.62:39305)
0619 08:53:51.207027 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:51.207036 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:51.207070 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:53:51.207200 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:51.207207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:51.207263 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:51.207421 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:51.208008 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.209470 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.210567 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.211172 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.211300 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.214062 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:51.214066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:51.214066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:51.214078 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:51.215985 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:51.216352 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:51.216355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:51.216355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:51.216454 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:51.216624 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:51.216624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:51.216625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:51.216925 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:51.217047 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:51.217490 (+ 443us) server_negotiation.cc:299] Negotiation successful
0619 08:53:51.217620 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":88,"threads_started":1}
I20260619 08:53:51.220974 31618 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:53:51.221462 31618 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:53:51.239619 31618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Generated new TSK 0
I20260619 08:53:51.239817 31618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:53:51.271826 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39305
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:51.381696 31634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:51.381917 31634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:51.381944 31634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:51.383023 31634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:51.383062 31634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:51.383090 31634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:53:51.384663 31634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:51.384910 31634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:51.385105 31634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:51.389931 31639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:51.389925 31640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:51.389923 31642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:51.390341 31634 server_base.cc:1034] running on GCE node
I20260619 08:53:51.390573 31634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:51.390811 31634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:51.391973 31634 hybrid_clock.cc:648] HybridClock initialized: now 1781859231391966 us; error 58 us; skew 500 ppm
Jun 19 08:53:51 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859231, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:51.394831 31634 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:53:51.395212 31634 webserver.cc:460] Webserver started at http://127.15.208.1:38683/ using document root <none> and password file <none>
I20260619 08:53:51.395386 31634 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:51.395427 31634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:51.395514 31634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:51.396528 31634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "3bae253cd56147078fee97b395038889"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "d5a8ff8c3aae0876a12c8adfb132c844"
server_key_iv: "e5d9189b5567f4434900190425dae64e"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.396914 31634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "3bae253cd56147078fee97b395038889"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "d5a8ff8c3aae0876a12c8adfb132c844"
server_key_iv: "e5d9189b5567f4434900190425dae64e"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.398293 31634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:53:51.399083 31649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.399264 31634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:53:51.399358 31634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "3bae253cd56147078fee97b395038889"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "d5a8ff8c3aae0876a12c8adfb132c844"
server_key_iv: "e5d9189b5567f4434900190425dae64e"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.399438 31634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:51.411083 31634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:51.413054 31634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:51.413178 31634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:51.413378 31634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:51.413690 31634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:51.413724 31634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.413748 31634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:51.413763 31634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.420077 31634 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:40283
I20260619 08:53:51.420153 31762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:40283 every 8 connection(s)
I20260619 08:53:51.420501 31634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:53:51 dist-test-slave-p0h9 krb5kdc[31527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859231, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:51.427616 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 31634
I20260619 08:53:51.427744 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:53:51.427994 16192 external_mini_cluster.cc:1428] Setting key ff82d5a61084225c8b06a0f59b18e26e
I20260619 08:53:51.430089 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.421286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:33177 (local address 127.15.208.62:39305)
0619 08:53:51.421361 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:51.421363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:51.421534 (+ 171us) server_negotiation.cc:407] Connection header received
0619 08:53:51.421996 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:51.421998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:51.422012 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:51.422084 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:51.423006 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.423539 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.424479 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.424680 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.424811 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.426909 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:51.426910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:51.426910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:51.426917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:51.428561 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:51.429036 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:51.429038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:51.429038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:51.429103 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:51.429355 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:51.429356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:51.429356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:51.429488 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:51.429569 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:51.429901 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 08:53:51.430002 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:53:51.430280 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.421239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:33177)
0619 08:53:51.421396 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:51.421438 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:51.421912 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:51.422137 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:51.422141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:51.422712 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:51.422909 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.422913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.423625 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.423626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.424394 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.424396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.424824 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.424825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.424928 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.425394 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:51.425403 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:51.426817 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:51.428660 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:51.428667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:51.428669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:51.428950 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:51.429176 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:51.429177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:51.429178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:51.429282 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:51.429562 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:51.429569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:51.429754 (+ 185us) client_negotiation.cc:769] Sending connection context
0619 08:53:51.429863 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 08:53:51.430024 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260619 08:53:51.430779 31763 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39305
I20260619 08:53:51.430891 31763 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:51.431106 31763 heartbeater.cc:511] Master 127.15.208.62:39305 requested a full tablet report, sending...
I20260619 08:53:51.431646 31572 ts_manager.cc:194] Registered new tserver with Master: 3bae253cd56147078fee97b395038889 (127.15.208.1:40283)
I20260619 08:53:51.432276 31572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:33177
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:53:51.452083 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39305
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:51.532212 31768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:51.532416 31768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:51.532440 31768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:51.533541 31768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:51.533581 31768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:51.533607 31768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:53:51.534957 31768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:51.535136 31768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:51.535312 31768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:51.540333 31773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:51.540297 31776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:51.540287 31774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:51.540282 31768 server_base.cc:1034] running on GCE node
I20260619 08:53:51.540690 31768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:51.541015 31768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:51.542158 31768 hybrid_clock.cc:648] HybridClock initialized: now 1781859231542141 us; error 35 us; skew 500 ppm
Jun 19 08:53:51 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859231, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:51.544559 31768 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:53:51.545091 31768 webserver.cc:460] Webserver started at http://127.15.208.2:38247/ using document root <none> and password file <none>
I20260619 08:53:51.545321 31768 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:51.545387 31768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:51.545531 31768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:51.546665 31768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "870fe4352c0c4c38bdeffeec60c8836a"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "c44e2fb1cef99dfa4c391f0514bdfbb5"
server_key_iv: "3db30fb4ff1b9e114b861af2de4018ba"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.547042 31768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "870fe4352c0c4c38bdeffeec60c8836a"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "c44e2fb1cef99dfa4c391f0514bdfbb5"
server_key_iv: "3db30fb4ff1b9e114b861af2de4018ba"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.548391 31768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:53:51.549225 31783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.549396 31768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.549527 31768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "870fe4352c0c4c38bdeffeec60c8836a"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "c44e2fb1cef99dfa4c391f0514bdfbb5"
server_key_iv: "3db30fb4ff1b9e114b861af2de4018ba"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.549604 31768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:51.563113 31768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:51.565042 31768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:51.565201 31768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:51.565441 31768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:51.565743 31768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:51.565806 31768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.565851 31768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:51.565881 31768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.571728 31768 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:43577
I20260619 08:53:51.571807 31896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:43577 every 8 connection(s)
I20260619 08:53:51.572176 31768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:53:51.576776 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 31768
I20260619 08:53:51.576910 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:53:51.577153 16192 external_mini_cluster.cc:1428] Setting key ee64059be4d3b7d06613352f3e97d19f
Jun 19 08:53:51 dist-test-slave-p0h9 krb5kdc[31527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859231, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:51.581591 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.572982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:56567 (local address 127.15.208.62:39305)
0619 08:53:51.573031 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:51.573034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:51.573316 (+ 282us) server_negotiation.cc:407] Connection header received
0619 08:53:51.573827 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:51.573828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:51.573838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:51.573892 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:51.574780 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.575269 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.576214 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.576392 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.576445 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.578675 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:51.578676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:51.578677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:51.578684 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:51.580088 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:51.580557 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:51.580559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:51.580559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:51.580622 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:51.580911 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:51.580912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:51.580913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:51.581065 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:51.581137 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:51.581380 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 08:53:51.581472 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:53:51.581746 31899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.572989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:56567)
0619 08:53:51.573164 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:51.573201 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:51.573742 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:51.573943 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:51.573946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:51.574500 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:51.574705 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.574710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.575346 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.575347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.576138 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.576140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.576463 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.576464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.576574 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.577180 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:51.577188 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:51.578586 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:51.580173 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:51.580179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:51.580181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:51.580448 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:51.580693 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:51.580693 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:51.580694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:51.580844 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:51.581132 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:51.581137 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:51.581249 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 08:53:51.581331 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:53:51.581450 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260619 08:53:51.582367 31897 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39305
I20260619 08:53:51.582459 31897 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:51.582677 31897 heartbeater.cc:511] Master 127.15.208.62:39305 requested a full tablet report, sending...
I20260619 08:53:51.583114 31572 ts_manager.cc:194] Registered new tserver with Master: 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577)
I20260619 08:53:51.583693 31572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:56567
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:53:51.601926 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39305
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:53:51.683197 31902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:53:51.683395 31902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:53:51.683418 31902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:53:51.684495 31902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:53:51.684530 31902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:53:51.684556 31902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:53:51.685930 31902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:53:51.686174 31902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:53:51.686378 31902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:53:51.691182 31907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:53:51.691315 31910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:51.691212 31902 server_base.cc:1034] running on GCE node
W20260619 08:53:51.691212 31908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:53:51.691639 31902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:53:51.691883 31902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:53:51.693049 31902 hybrid_clock.cc:648] HybridClock initialized: now 1781859231693038 us; error 40 us; skew 500 ppm
Jun 19 08:53:51 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859231, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:53:51.695533 31902 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:53:51.695971 31902 webserver.cc:460] Webserver started at http://127.15.208.3:39571/ using document root <none> and password file <none>
I20260619 08:53:51.696166 31902 fs_manager.cc:359] Metadata directory not provided
I20260619 08:53:51.696208 31902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:53:51.696342 31902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:53:51.697398 31902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "93eb0670d25443678399215ef2336242"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "a9f1c94b6d99923917f7a0a4947b932c"
server_key_iv: "87bb0a24b8ed55f2527dfc1e06dbec7f"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.697700 31902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "93eb0670d25443678399215ef2336242"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "a9f1c94b6d99923917f7a0a4947b932c"
server_key_iv: "87bb0a24b8ed55f2527dfc1e06dbec7f"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.698886 31902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:53:51.699551 31917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.699738 31902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:53:51.699819 31902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "93eb0670d25443678399215ef2336242"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "a9f1c94b6d99923917f7a0a4947b932c"
server_key_iv: "87bb0a24b8ed55f2527dfc1e06dbec7f"
server_key_version: "encryptionkey@0"
I20260619 08:53:51.699872 31902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:53:51.714710 31902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:53:51.716499 31902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:53:51.716643 31902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:53:51.716931 31902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:53:51.717252 31902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:53:51.717285 31902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.717308 31902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:53:51.717365 31902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:53:51.722889 31902 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:41589
I20260619 08:53:51.722970 32030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:41589 every 8 connection(s)
I20260619 08:53:51.723291 31902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:53:51.726943 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 31902
I20260619 08:53:51.727061 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:53:51.727248 16192 external_mini_cluster.cc:1428] Setting key 83dbe36147b3b8133ddd8a8ebe51b906
Jun 19 08:53:51 dist-test-slave-p0h9 krb5kdc[31527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859231, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:53:51.732666 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.723997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:42307 (local address 127.15.208.62:39305)
0619 08:53:51.724054 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:51.724056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:51.724240 (+ 184us) server_negotiation.cc:407] Connection header received
0619 08:53:51.724759 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:51.724761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:51.724772 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:51.724835 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:51.725790 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.726283 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.727668 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.727956 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.728015 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.730279 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:51.730280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:51.730281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:51.730288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:51.731551 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:51.731910 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:51.731911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:51.731911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:51.731960 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:51.732169 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:51.732169 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:51.732170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:51.732267 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:51.732315 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:51.732543 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 08:53:51.732603 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:53:51.732932 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.723950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:42307)
0619 08:53:51.724112 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:51.724154 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:51.724646 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:51.724912 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:51.724915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:51.725507 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:53:51.725699 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.725703 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.726383 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.726384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.727574 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.727577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.728051 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.728052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.728222 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.728697 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:53:51.728705 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:53:51.730157 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:53:51.731624 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:51.731630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:51.731631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:51.731845 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:51.732019 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:53:51.732020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:53:51.732020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:53:51.732115 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:53:51.732321 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:53:51.732325 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:53:51.732433 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 08:53:51.732512 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:53:51.732644 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260619 08:53:51.733434 32031 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39305
I20260619 08:53:51.733564 32031 heartbeater.cc:464] Registering TS with master...
I20260619 08:53:51.733785 32031 heartbeater.cc:511] Master 127.15.208.62:39305 requested a full tablet report, sending...
I20260619 08:53:51.734248 31572 ts_manager.cc:194] Registered new tserver with Master: 93eb0670d25443678399215ef2336242 (127.15.208.3:41589)
I20260619 08:53:51.734647 31572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:42307
I20260619 08:53:51.739130 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:53:51.746428 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.740576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.15.208.62:39305)
0619 08:53:51.740649 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:51.740651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:51.740781 (+ 130us) server_negotiation.cc:407] Connection header received
0619 08:53:51.740915 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:51.740916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:51.740925 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:51.740974 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:53:51.741469 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.741923 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.742620 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.742791 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.742843 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.744035 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:53:51.744037 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:53:51.744037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:53:51.744043 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:53:51.745445 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:51.745767 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:51.745768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:51.745768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:51.745816 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:53:51.745992 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:53:51.745992 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:53:51.745992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:53:51.746132 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:53:51.746207 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:51.746291 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 08:53:51.746345 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:53:51.748180 31572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:53:51.749799 31572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:53:51.756500 32042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.752085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:41589 (local address 127.0.0.1:42732)
0619 08:53:51.752473 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:51.752516 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:51.752624 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:51.752969 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:51.752972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:51.752977 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:51.753047 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.753049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.754035 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.754036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.755258 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.755261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.756191 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.756192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.756284 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.756299 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 08:53:51.756372 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:53:51.756422 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":8,"thread_start_us":63,"threads_started":1}
I20260619 08:53:51.756608 32043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.752085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:43577 (local address 127.0.0.1:39660)
0619 08:53:51.752474 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:51.752516 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:51.752623 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:51.752969 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:51.752972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:51.752974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:51.753058 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.753060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.754372 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.754373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.755395 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.755397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.756337 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.756338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.756424 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.756429 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:53:51.756481 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:53:51.756522 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260619 08:53:51.756918 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.752206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.15.208.2:43577)
0619 08:53:51.752405 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:51.752411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:51.752583 (+ 172us) server_negotiation.cc:407] Connection header received
0619 08:53:51.752794 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:51.752796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:51.752813 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:51.752888 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:51.753139 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.754237 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.755485 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.756193 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.756253 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.756306 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:51.756804 (+ 498us) server_negotiation.cc:299] Negotiation successful
0619 08:53:51.756850 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":80,"threads_started":1}
I20260619 08:53:51.757337 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.752356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:40283 (local address 127.0.0.1:44220)
0619 08:53:51.752624 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:51.752635 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:51.752690 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:51.753021 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:51.753022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:51.753024 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:51.753071 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.753072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.755480 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.755480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.756119 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.756120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.757094 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.757095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.757160 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.757165 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:53:51.757218 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:53:51.757257 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":133,"threads_started":1}
I20260619 08:53:51.756536 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.752204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.15.208.3:41589)
0619 08:53:51.752425 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:51.752429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:51.752581 (+ 152us) server_negotiation.cc:407] Connection header received
0619 08:53:51.752794 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:51.752796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:51.752813 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:51.752888 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:51.753155 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.753926 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.755343 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.756099 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.756178 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.756242 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:51.756399 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 08:53:51.756453 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":97,"threads_started":1}
I20260619 08:53:51.757606 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.752449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44220 (local address 127.15.208.1:40283)
0619 08:53:51.752607 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:51.752613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:51.752770 (+ 157us) server_negotiation.cc:407] Connection header received
0619 08:53:51.752818 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:51.752820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:51.752837 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:51.752895 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:51.753167 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.754243 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.756232 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.757003 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.757087 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.757144 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:51.757513 (+ 369us) server_negotiation.cc:299] Negotiation successful
0619 08:53:51.757550 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260619 08:53:51.757681 31831 tablet_service.cc:1467] Processing CreateTablet for tablet 5edab553e34f4527be03d968d93aaf40 (DEFAULT_TABLE table=test-table [id=cc6154caaec3480ca09daead249ba606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:51.758041 31831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5edab553e34f4527be03d968d93aaf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:51.758036 31965 tablet_service.cc:1467] Processing CreateTablet for tablet 5edab553e34f4527be03d968d93aaf40 (DEFAULT_TABLE table=test-table [id=cc6154caaec3480ca09daead249ba606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:51.758157 31697 tablet_service.cc:1467] Processing CreateTablet for tablet 5edab553e34f4527be03d968d93aaf40 (DEFAULT_TABLE table=test-table [id=cc6154caaec3480ca09daead249ba606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:53:51.758330 31965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5edab553e34f4527be03d968d93aaf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:51.758396 31697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5edab553e34f4527be03d968d93aaf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:53:51.761368 32049 tablet_bootstrap.cc:492] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: Bootstrap starting.
I20260619 08:53:51.761711 32048 tablet_bootstrap.cc:492] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Bootstrap starting.
I20260619 08:53:51.762018 32049 tablet_bootstrap.cc:654] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:51.762216 32048 tablet_bootstrap.cc:654] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:51.762586 32049 log.cc:824] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:51.762662 32048 log.cc:824] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:51.762984 32050 tablet_bootstrap.cc:492] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: Bootstrap starting.
I20260619 08:53:51.763386 32049 tablet_bootstrap.cc:492] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: No bootstrap required, opened a new log
I20260619 08:53:51.763461 32048 tablet_bootstrap.cc:492] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: No bootstrap required, opened a new log
I20260619 08:53:51.763470 32049 ts_tablet_manager.cc:1397] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:53:51.763538 32048 ts_tablet_manager.cc:1397] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:53:51.763608 32050 tablet_bootstrap.cc:654] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: Neither blocks nor log segments found. Creating new log.
I20260619 08:53:51.764119 32050 log.cc:824] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: Log is configured to *not* fsync() on all Append() calls
I20260619 08:53:51.764904 32050 tablet_bootstrap.cc:492] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: No bootstrap required, opened a new log
I20260619 08:53:51.764982 32050 ts_tablet_manager.cc:1397] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:53:51.765225 32049 raft_consensus.cc:348] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:53:51.765377 32049 raft_consensus.cc:374] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:51.765420 32049 raft_consensus.cc:729] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 870fe4352c0c4c38bdeffeec60c8836a, State: Initialized, Role: FOLLOWER
I20260619 08:53:51.765573 32049 consensus_queue.cc:260] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:53:51.765844 32049 ts_tablet_manager.cc:1428] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:53:51.765859 31897 heartbeater.cc:503] Master 127.15.208.62:39305 was elected leader, sending a full tablet report...
I20260619 08:53:51.765957 32048 raft_consensus.cc:348] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:53:51.766131 32048 raft_consensus.cc:374] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:51.766165 32048 raft_consensus.cc:729] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bae253cd56147078fee97b395038889, State: Initialized, Role: FOLLOWER
I20260619 08:53:51.766278 32048 consensus_queue.cc:260] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:53:51.766552 32048 ts_tablet_manager.cc:1428] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:53:51.766568 31763 heartbeater.cc:503] Master 127.15.208.62:39305 was elected leader, sending a full tablet report...
I20260619 08:53:51.766947 32050 raft_consensus.cc:348] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:53:51.767131 32050 raft_consensus.cc:374] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:53:51.767171 32050 raft_consensus.cc:729] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93eb0670d25443678399215ef2336242, State: Initialized, Role: FOLLOWER
I20260619 08:53:51.767293 32050 consensus_queue.cc:260] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:53:51.767560 32031 heartbeater.cc:503] Master 127.15.208.62:39305 was elected leader, sending a full tablet report...
I20260619 08:53:51.767555 32050 ts_tablet_manager.cc:1428] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260619 08:53:51.822863 31898 tablet.cc:2369] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:53:51.921372 31764 tablet.cc:2369] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:53:51.973845 32032 tablet.cc:2369] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:53:51.995335 32056 raft_consensus.cc:482] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:53:51.995463 32056 raft_consensus.cc:504] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:53:51.995888 32056 leader_election.cc:288] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577), 3bae253cd56147078fee97b395038889 (127.15.208.1:40283)
I20260619 08:53:51.998816 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.996130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:43577 (local address 127.15.208.3:34827)
0619 08:53:51.996187 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:51.996201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:51.996295 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:51.996432 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:51.996433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:51.996435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:51.996522 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.996523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.997235 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.997236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.997969 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.997971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.998539 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.998540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.998638 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.998641 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:53:51.998707 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 08:53:51.998748 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1}
I20260619 08:53:51.998816 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.996239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:34827 (local address 127.15.208.2:43577)
0619 08:53:51.996308 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:51.996311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:51.996321 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:53:51.996355 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:51.996356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:51.996365 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:51.996422 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:51.996600 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.997143 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.998069 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.998447 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.998534 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.998542 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:51.998713 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 08:53:51.998748 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:53:51.998942 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.996264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:40283 (local address 127.15.208.3:50409)
0619 08:53:51.996440 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:51.996454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:53:51.996528 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:53:51.996658 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:53:51.996658 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:53:51.996660 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:53:51.996757 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.996759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.997391 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.997392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.998123 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:53:51.998125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.998680 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.998681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:53:51.998755 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.998763 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 08:53:51.998830 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 08:53:51.998865 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260619 08:53:51.998934 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:51.996352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:50409 (local address 127.15.208.1:40283)
0619 08:53:51.996395 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:51.996397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:51.996508 (+ 111us) server_negotiation.cc:407] Connection header received
0619 08:53:51.996581 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:51.996582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:51.996595 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:51.996647 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:53:51.996812 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.997295 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.998213 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:51.998577 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:51.998681 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:51.998688 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:51.998834 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 08:53:51.998872 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:53:51.999260 31717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5edab553e34f4527be03d968d93aaf40" candidate_uuid: "93eb0670d25443678399215ef2336242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bae253cd56147078fee97b395038889" is_pre_election: true
I20260619 08:53:51.999256 31851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5edab553e34f4527be03d968d93aaf40" candidate_uuid: "93eb0670d25443678399215ef2336242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "870fe4352c0c4c38bdeffeec60c8836a" is_pre_election: true
I20260619 08:53:51.999410 31717 raft_consensus.cc:2454] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93eb0670d25443678399215ef2336242 in term 0.
I20260619 08:53:51.999410 31851 raft_consensus.cc:2454] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93eb0670d25443678399215ef2336242 in term 0.
I20260619 08:53:51.999624 31919 leader_election.cc:302] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bae253cd56147078fee97b395038889, 93eb0670d25443678399215ef2336242; no voters:
I20260619 08:53:51.999785 32056 raft_consensus.cc:2781] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:53:51.999859 32056 raft_consensus.cc:482] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:53:51.999877 32056 raft_consensus.cc:3037] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:52.000578 32056 raft_consensus.cc:504] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:53:52.000706 32056 leader_election.cc:288] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [CANDIDATE]: Term 1 election: Requested vote from peers 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577), 3bae253cd56147078fee97b395038889 (127.15.208.1:40283)
I20260619 08:53:52.000946 31851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5edab553e34f4527be03d968d93aaf40" candidate_uuid: "93eb0670d25443678399215ef2336242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "870fe4352c0c4c38bdeffeec60c8836a"
I20260619 08:53:52.000957 31717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5edab553e34f4527be03d968d93aaf40" candidate_uuid: "93eb0670d25443678399215ef2336242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bae253cd56147078fee97b395038889"
I20260619 08:53:52.001044 31851 raft_consensus.cc:3037] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:52.001044 31717 raft_consensus.cc:3037] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:53:52.001933 31851 raft_consensus.cc:2454] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93eb0670d25443678399215ef2336242 in term 1.
I20260619 08:53:52.001948 31717 raft_consensus.cc:2454] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93eb0670d25443678399215ef2336242 in term 1.
I20260619 08:53:52.002126 31919 leader_election.cc:302] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 870fe4352c0c4c38bdeffeec60c8836a, 93eb0670d25443678399215ef2336242; no voters:
I20260619 08:53:52.002223 32056 raft_consensus.cc:2781] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:53:52.002403 32056 raft_consensus.cc:686] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 1 LEADER]: Becoming Leader. State: Replica: 93eb0670d25443678399215ef2336242, State: Running, Role: LEADER
I20260619 08:53:52.002507 32056 consensus_queue.cc:237] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:53:52.003425 31572 catalog_manager.cc:5495] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93eb0670d25443678399215ef2336242 (127.15.208.3). New cstate: current_term: 1 leader_uuid: "93eb0670d25443678399215ef2336242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } health_report { overall_health: UNKNOWN } } }
I20260619 08:53:52.039084 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:52.036916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42736 (local address 127.15.208.3:41589)
0619 08:53:52.036972 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:53:52.036986 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 08:53:52.036998 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:53:52.037041 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:53:52.037042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:53:52.037060 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:53:52.037124 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:53:52.037249 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:52.037716 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:52.038407 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:53:52.038587 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:53:52.038634 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:53:52.038769 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:53:52.038867 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:53:52.038926 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:53:52.038993 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 08:53:52.039010 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:53:52.041076 31717 raft_consensus.cc:1261] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Refusing update from remote peer 93eb0670d25443678399215ef2336242: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:53:52.041075 31851 raft_consensus.cc:1261] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a [term 1 FOLLOWER]: Refusing update from remote peer 93eb0670d25443678399215ef2336242: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:53:52.041469 32058 consensus_queue.cc:1035] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:53:52.041615 32056 consensus_queue.cc:1035] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:53:52.044631 32063 mvcc.cc:205] Tried to move back new op lower bound from 7298495414437007360 to 7298495414282907648. Current Snapshot: MvccSnapshot[applied={T|T < 7298495414437007360}]
I20260619 08:53:52.046257 31571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:53:52.046355 31571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:53:52.047134 31571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Sending DeleteTablet for 3 replicas of tablet 5edab553e34f4527be03d968d93aaf40
I20260619 08:53:52.047432 31965 tablet_service.cc:1514] Processing DeleteTablet for tablet 5edab553e34f4527be03d968d93aaf40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:52 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:42732
I20260619 08:53:52.047466 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 31634
I20260619 08:53:52.047539 31831 tablet_service.cc:1514] Processing DeleteTablet for tablet 5edab553e34f4527be03d968d93aaf40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:52 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:39660
I20260619 08:53:52.048254 32072 tablet_replica.cc:330] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: stopping tablet replica
I20260619 08:53:52.048318 32071 tablet_replica.cc:330] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: stopping tablet replica
I20260619 08:53:52.048416 32072 raft_consensus.cc:2229] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:53:52.048424 32071 raft_consensus.cc:2229] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:53:52.048537 32071 raft_consensus.cc:2258] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:52.048580 32072 raft_consensus.cc:2258] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:53:52.049033 32072 ts_tablet_manager.cc:1905] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:53:52.049033 32071 ts_tablet_manager.cc:1905] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:53:52.050657 32072 ts_tablet_manager.cc:1918] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:53:52.050670 32071 ts_tablet_manager.cc:1918] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:53:52.050731 32071 log.cc:1196] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/5edab553e34f4527be03d968d93aaf40
I20260619 08:53:52.050819 32072 log.cc:1196] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/5edab553e34f4527be03d968d93aaf40
I20260619 08:53:52.051048 32071 ts_tablet_manager.cc:1939] T 5edab553e34f4527be03d968d93aaf40 P 93eb0670d25443678399215ef2336242: Deleting consensus metadata
I20260619 08:53:52.051172 32072 ts_tablet_manager.cc:1939] T 5edab553e34f4527be03d968d93aaf40 P 870fe4352c0c4c38bdeffeec60c8836a: Deleting consensus metadata
I20260619 08:53:52.051515 31556 catalog_manager.cc:4841] TS 93eb0670d25443678399215ef2336242 (127.15.208.3:41589): tablet 5edab553e34f4527be03d968d93aaf40 (table test-table [id=cc6154caaec3480ca09daead249ba606]) successfully deleted
I20260619 08:53:52.051671 31557 catalog_manager.cc:4841] TS 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577): tablet 5edab553e34f4527be03d968d93aaf40 (table test-table [id=cc6154caaec3480ca09daead249ba606]) successfully deleted
W20260619 08:53:52.054296 31557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:40283 (error 108)
I20260619 08:53:52.054455 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 31768
I20260619 08:53:52.055006 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:53:52.054628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:40283 (local address 127.0.0.1:44230)
0619 08:53:52.054935 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 08:53:52.054968 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:40283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289}
W20260619 08:53:52.055255 31557 catalog_manager.cc:4584] TS 3bae253cd56147078fee97b395038889 (127.15.208.1:40283): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5edab553e34f4527be03d968d93aaf40: Network error: Client connection negotiation failed: client connection to 127.15.208.1:40283: connect: Connection refused (error 111)
I20260619 08:53:52.059859 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 31902
I20260619 08:53:52.064298 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 31540
I20260619 08:54:12.069658 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39305
--webserver_interface=127.15.208.62
--webserver_port=32787
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:39305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:12.150768 32074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:12.150974 32074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:12.150996 32074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:12.152076 32074 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 08:54:12.152110 32074 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 08:54:12.152123 32074 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 08:54:12.152134 32074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:54:12.152148 32074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:12.152159 32074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:12.152171 32074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:54:12.152184 32074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:54:12.153551 32074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:39305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=32787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:12.153735 32074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:12.153895 32074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:12.162680 32080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:12.162592 32081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:12.162878 32083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:12.162930 32074 server_base.cc:1034] running on GCE node
I20260619 08:54:12.163239 32074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:12.163601 32074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:12.164795 32074 hybrid_clock.cc:648] HybridClock initialized: now 1781859252164774 us; error 48 us; skew 500 ppm
Jun 19 08:54:12 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859252, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:12.167330 32074 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:54:12.167748 32074 webserver.cc:460] Webserver started at http://127.15.208.62:32787/ using document root <none> and password file <none>
I20260619 08:54:12.167955 32074 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:12.168007 32074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:12.169411 32074 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:12.170082 32090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:12.170282 32074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:54:12.170365 32074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "3521573a48134e558acf607515f56deb"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "6d63c4561cfcc4fdd1c7ce6bcce252a9"
server_key_iv: "08fdb93dfa7fffa7f9a6f16c98dc82cf"
server_key_version: "encryptionkey@0"
I20260619 08:54:12.170748 32074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:12.187327 32074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:12.189713 32074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:12.189833 32074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:12.193840 32074 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:39305
I20260619 08:54:12.193918 32142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:39305 every 8 connection(s)
I20260619 08:54:12.194330 32074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:54:12.194805 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 32074
I20260619 08:54:12.195153 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:40283
--local_ip_for_outbound_sockets=127.15.208.1
--tserver_master_addrs=127.15.208.62:39305
--webserver_port=38683
--webserver_interface=127.15.208.1
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:54:12.196648 32143 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:54:12.197356 32143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Bootstrap starting.
I20260619 08:54:12.205991 32143 log.cc:824] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:12.208109 32143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:54:12.208442 32143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Bootstrap complete.
I20260619 08:54:12.210879 32143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:54:12.211211 32143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3521573a48134e558acf607515f56deb, State: Initialized, Role: FOLLOWER
I20260619 08:54:12.211386 32143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:54:12.211447 32143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:54:12.211516 32143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:54:12.211575 32143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:54:12.212596 32143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:54:12.212683 32143 leader_election.cc:302] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3521573a48134e558acf607515f56deb; no voters:
I20260619 08:54:12.212909 32143 leader_election.cc:288] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:54:12.212949 32147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:54:12.213070 32147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 2 LEADER]: Becoming Leader. State: Replica: 3521573a48134e558acf607515f56deb, State: Running, Role: LEADER
I20260619 08:54:12.213203 32143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:54:12.213263 32147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:54:12.213562 32147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3521573a48134e558acf607515f56deb. Latest consensus state: current_term: 2 leader_uuid: "3521573a48134e558acf607515f56deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } } }
I20260619 08:54:12.213567 32148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3521573a48134e558acf607515f56deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } } }
I20260619 08:54:12.213791 32148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:12.214087 32147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:12.214170 32156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:54:12.215067 32156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:54:12.215301 32156 catalog_manager.cc:1259] Loaded cluster ID: 6b2413d248de4759abf85340654d2686
I20260619 08:54:12.215376 32156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:54:12.216117 32156 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:54:12.216238 32156 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Loaded TSK: 0
I20260619 08:54:12.225353 32156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Generated new TSK 1
I20260619 08:54:12.225472 32156 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:54:12.287740 32145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:12.287982 32145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:12.288020 32145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:12.289206 32145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:12.289276 32145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:12.289322 32145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:54:12.290853 32145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:40283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=38683
--tserver_master_addrs=127.15.208.62:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:12.291144 32145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:12.291399 32145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:12.296363 32170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:12.296360 32171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:12.296442 32173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:12.296761 32145 server_base.cc:1034] running on GCE node
I20260619 08:54:12.296962 32145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:12.297219 32145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:12.298400 32145 hybrid_clock.cc:648] HybridClock initialized: now 1781859252298369 us; error 34 us; skew 500 ppm
Jun 19 08:54:12 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859252, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:12.301148 32145 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:54:12.301635 32145 webserver.cc:460] Webserver started at http://127.15.208.1:38683/ using document root <none> and password file <none>
I20260619 08:54:12.301885 32145 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:12.301972 32145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:12.303471 32145 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:12.304199 32180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:12.304383 32145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:12.304497 32145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "3bae253cd56147078fee97b395038889"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "d5a8ff8c3aae0876a12c8adfb132c844"
server_key_iv: "e5d9189b5567f4434900190425dae64e"
server_key_version: "encryptionkey@0"
I20260619 08:54:12.304904 32145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:12.321074 32145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:12.323324 32145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:12.323475 32145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:12.323740 32145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:12.324319 32187 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:54:12.326313 32145 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:54:12.326377 32145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:54:12.326419 32145 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:54:12.327052 32145 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:54:12.327114 32145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:54:12.327183 32187 tablet_bootstrap.cc:492] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Bootstrap starting.
I20260619 08:54:12.334623 32145 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:40283
I20260619 08:54:12.334739 32294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:40283 every 8 connection(s)
I20260619 08:54:12.335072 32145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:54:12.339190 32187 log.cc:824] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:12.341074 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 32145
I20260619 08:54:12.341378 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:43577
--local_ip_for_outbound_sockets=127.15.208.2
--tserver_master_addrs=127.15.208.62:39305
--webserver_port=38247
--webserver_interface=127.15.208.2
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:54:12.342347 32187 tablet_bootstrap.cc:492] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:54:12.342684 32187 tablet_bootstrap.cc:492] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Bootstrap complete.
Jun 19 08:54:12 dist-test-slave-p0h9 krb5kdc[31527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859252, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:12.343902 32187 ts_tablet_manager.cc:1397] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260619 08:54:12.345161 32187 raft_consensus.cc:348] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.345456 32187 raft_consensus.cc:729] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bae253cd56147078fee97b395038889, State: Initialized, Role: FOLLOWER
I20260619 08:54:12.345613 32187 consensus_queue.cc:260] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.345958 32187 ts_tablet_manager.cc:1428] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:54:12.347430 32297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.335858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:46085)
0619 08:54:12.336073 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:12.336122 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:12.336601 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:12.337149 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:12.337153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:12.337436 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:12.337657 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.337662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.338848 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.338849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.339995 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.339998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.340802 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.340803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.340994 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.341629 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:12.341639 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:12.343528 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:12.345636 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:12.345656 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:12.345659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:12.346019 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:12.346332 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:12.346333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:12.346333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:12.346459 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:12.346948 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:12.346954 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:12.347076 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 08:54:12.347163 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 08:54:12.347291 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
I20260619 08:54:12.347577 32298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.335938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:46085 (local address 127.15.208.62:39305)
0619 08:54:12.336160 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.336170 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.336206 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:54:12.336875 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.336882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.336960 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.337118 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:12.337788 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.338700 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.340082 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.340651 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.340797 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.343646 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:12.343650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:12.343651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:12.343662 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:12.345477 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.346125 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.346129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.346129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.346252 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.346558 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.346559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.346559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.346862 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:12.346956 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.347243 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.347381 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":115,"threads_started":1}
I20260619 08:54:12.348415 32295 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39305
I20260619 08:54:12.348512 32295 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:12.348687 32295 heartbeater.cc:511] Master 127.15.208.62:39305 requested a full tablet report, sending...
I20260619 08:54:12.349475 32107 ts_manager.cc:194] Registered new tserver with Master: 3bae253cd56147078fee97b395038889 (127.15.208.1:40283)
I20260619 08:54:12.350677 32107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:46085
I20260619 08:54:12.351686 32295 heartbeater.cc:503] Master 127.15.208.62:39305 was elected leader, sending a full tablet report...
Jun 19 08:54:12 dist-test-slave-p0h9 krb5kdc[31527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859252, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.1@KRBTEST.COM
I20260619 08:54:12.359944 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.350166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:40283 (local address 127.0.0.1:49830)
0619 08:54:12.350395 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:12.350430 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:12.350514 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:12.350731 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:12.350738 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:12.351132 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:12.351225 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.351227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.352371 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.352372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.353361 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.353365 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.353814 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.353815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.353905 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.354975 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:12.354980 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:12.356456 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:12.358616 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:12.358624 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:12.358626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:12.358836 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:12.359159 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:12.359160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:12.359160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:12.359249 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:12.359556 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:12.359564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:12.359618 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 08:54:12.359720 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 08:54:12.359844 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":90,"threads_started":1}
I20260619 08:54:12.359944 32303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.350222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.15.208.1:40283)
0619 08:54:12.350374 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.350379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.350512 (+ 133us) server_negotiation.cc:407] Connection header received
0619 08:54:12.350617 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.350619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.350633 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.350709 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:12.351314 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.352292 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.353444 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.353724 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.353780 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.356561 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:12.356570 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:12.356570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:12.356584 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:12.358509 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.358985 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.358988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.358988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.359073 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.359322 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.359323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.359323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.359485 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:12.359578 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.359757 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.359853 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260619 08:54:12.360805 32229 tablet_service.cc:1514] Processing DeleteTablet for tablet 5edab553e34f4527be03d968d93aaf40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:53:52 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:49830
I20260619 08:54:12.361334 32304 tablet_replica.cc:330] stopping tablet replica
I20260619 08:54:12.361464 32304 raft_consensus.cc:2229] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:12.361548 32304 raft_consensus.cc:2258] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:12.361855 32304 ts_tablet_manager.cc:1905] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:12.363831 32304 ts_tablet_manager.cc:1918] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:12.363909 32304 log.cc:1196] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal/wals/5edab553e34f4527be03d968d93aaf40
I20260619 08:54:12.364153 32304 ts_tablet_manager.cc:1939] T 5edab553e34f4527be03d968d93aaf40 P 3bae253cd56147078fee97b395038889: Deleting consensus metadata
I20260619 08:54:12.364885 32092 catalog_manager.cc:4841] TS 3bae253cd56147078fee97b395038889 (127.15.208.1:40283): tablet 5edab553e34f4527be03d968d93aaf40 (table test-table [id=cc6154caaec3480ca09daead249ba606]) successfully deleted
W20260619 08:54:12.439579 32300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:12.439787 32300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:12.439810 32300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:12.441100 32300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:12.441146 32300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:12.441174 32300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:54:12.442641 32300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:43577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=38247
--tserver_master_addrs=127.15.208.62:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:12.442899 32300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:12.443157 32300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:12.448173 32312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:12.448187 32310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:12.448202 32309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:12.448606 32300 server_base.cc:1034] running on GCE node
I20260619 08:54:12.448875 32300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:12.449136 32300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:12.450289 32300 hybrid_clock.cc:648] HybridClock initialized: now 1781859252450277 us; error 30 us; skew 500 ppm
Jun 19 08:54:12 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859252, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:12.452941 32300 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:54:12.453336 32300 webserver.cc:460] Webserver started at http://127.15.208.2:38247/ using document root <none> and password file <none>
I20260619 08:54:12.453529 32300 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:12.453581 32300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:12.454864 32300 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:12.455423 32319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:12.455611 32300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:54:12.455680 32300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "870fe4352c0c4c38bdeffeec60c8836a"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "c44e2fb1cef99dfa4c391f0514bdfbb5"
server_key_iv: "3db30fb4ff1b9e114b861af2de4018ba"
server_key_version: "encryptionkey@0"
I20260619 08:54:12.456051 32300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:12.473981 32300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:12.476337 32300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:12.476450 32300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:12.476641 32300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:12.476986 32300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:12.477046 32300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:12.477087 32300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:12.477134 32300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:12.483549 32300 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:43577
I20260619 08:54:12.483593 32432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:43577 every 8 connection(s)
I20260619 08:54:12.483934 32300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:54:12 dist-test-slave-p0h9 krb5kdc[31527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859252, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:12.492184 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 32300
I20260619 08:54:12.492483 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:41589
--local_ip_for_outbound_sockets=127.15.208.3
--tserver_master_addrs=127.15.208.62:39305
--webserver_port=39571
--webserver_interface=127.15.208.3
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:54:12.493458 32298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.484631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:53673 (local address 127.15.208.62:39305)
0619 08:54:12.484696 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.484698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.484997 (+ 299us) server_negotiation.cc:407] Connection header received
0619 08:54:12.485521 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.485523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.485539 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.485608 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:12.486570 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.487072 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.488214 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.488532 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.488607 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.490739 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:12.490740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:12.490740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:12.490747 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:12.491957 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.492364 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.492365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.492366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.492433 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.492786 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.492787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.492788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.492914 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:12.492989 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.493265 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.493367 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:54:12.493613 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.484666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:53673)
0619 08:54:12.484868 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:12.484904 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:12.485399 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:12.485684 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:12.485687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:12.486273 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:12.486473 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.486477 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.487157 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.487158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.488123 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.488126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.488617 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.488618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.488808 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.489313 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:12.489321 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:12.490657 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:12.492031 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:12.492038 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:12.492039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:12.492253 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:12.492548 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:12.492549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:12.492550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:12.492697 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:12.492998 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:12.493005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:12.493131 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 08:54:12.493231 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 08:54:12.493386 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":121,"threads_started":1}
I20260619 08:54:12.494114 32433 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39305
I20260619 08:54:12.494231 32433 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:12.494407 32433 heartbeater.cc:511] Master 127.15.208.62:39305 requested a full tablet report, sending...
I20260619 08:54:12.494796 32107 ts_manager.cc:194] Registered new tserver with Master: 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577)
I20260619 08:54:12.495342 32107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:53673
W20260619 08:54:12.574847 32436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:12.575067 32436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:12.575093 32436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:12.576210 32436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:12.576254 32436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:12.576280 32436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:54:12.577731 32436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:41589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=39571
--tserver_master_addrs=127.15.208.62:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:12.578030 32436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:12.578267 32436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:12.583354 32444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:12.583362 32442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:12.583448 32436 server_base.cc:1034] running on GCE node
W20260619 08:54:12.583361 32441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:12.583855 32436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:12.584105 32436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:12.585284 32436 hybrid_clock.cc:648] HybridClock initialized: now 1781859252585264 us; error 38 us; skew 500 ppm
Jun 19 08:54:12 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859252, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:12.587779 32436 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:54:12.588233 32436 webserver.cc:460] Webserver started at http://127.15.208.3:39571/ using document root <none> and password file <none>
I20260619 08:54:12.588498 32436 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:12.588596 32436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:12.590932 32436 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260619 08:54:12.591544 32451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:12.591717 32436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:54:12.591815 32436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "93eb0670d25443678399215ef2336242"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "a9f1c94b6d99923917f7a0a4947b932c"
server_key_iv: "87bb0a24b8ed55f2527dfc1e06dbec7f"
server_key_version: "encryptionkey@0"
I20260619 08:54:12.592182 32436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:12.606695 32436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:12.608765 32436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:12.608940 32436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:12.609174 32436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:12.609493 32436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:12.609545 32436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:12.609580 32436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:12.609607 32436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:12.615754 32436 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:41589
I20260619 08:54:12.615805 32564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:41589 every 8 connection(s)
I20260619 08:54:12.616135 32436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:54:12.618487 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 32436
Jun 19 08:54:12 dist-test-slave-p0h9 krb5kdc[31527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859252, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:12.626592 32569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.619559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41748 (local address 127.15.208.62:39305)
0619 08:54:12.619685 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.619687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.619694 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:54:12.619720 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.619721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.619729 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.619784 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:12.620560 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.621212 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.622259 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.622431 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.622483 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.623733 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:12.623735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:12.623735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:12.623742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:12.625442 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.625781 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.625782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.625783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.625839 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.626049 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.626050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.626050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.626178 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:12.626364 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.626402 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.626483 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260619 08:54:12.627008 32298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.616849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:43141 (local address 127.15.208.62:39305)
0619 08:54:12.616909 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.616911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.617155 (+ 244us) server_negotiation.cc:407] Connection header received
0619 08:54:12.617637 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.617638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.617649 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.617701 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:12.618809 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.619441 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.620831 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.621105 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.621170 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.623684 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:12.623685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:12.623685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:12.623699 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:12.625355 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.625757 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.625758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.625759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.625819 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.626113 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.626114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.626114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.626225 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:12.626295 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.626580 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.626871 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:54:12.627089 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.616870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:43141)
0619 08:54:12.617052 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:12.617087 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:12.617547 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:12.617753 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:12.617756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:12.618400 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:12.618697 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.618704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.619540 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.619541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.620771 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.620774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.621178 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.621179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.621315 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.621829 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:12.621838 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:12.623581 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:12.625450 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:12.625457 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:12.625459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:12.625681 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:12.625886 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:12.625887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:12.625887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:12.626023 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:12.626300 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:12.626306 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:12.626442 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 08:54:12.626544 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 08:54:12.626820 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":103,"threads_started":1}
I20260619 08:54:12.627700 32565 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39305
I20260619 08:54:12.627781 32565 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:12.627921 32565 heartbeater.cc:511] Master 127.15.208.62:39305 requested a full tablet report, sending...
I20260619 08:54:12.628240 32107 ts_manager.cc:194] Registered new tserver with Master: 93eb0670d25443678399215ef2336242 (127.15.208.3:41589)
I20260619 08:54:12.628599 32107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:43141
I20260619 08:54:12.638757 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:54:12.642378 32298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.639448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41754 (local address 127.15.208.62:39305)
0619 08:54:12.639553 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.639555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.639602 (+ 47us) server_negotiation.cc:407] Connection header received
0619 08:54:12.639684 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.639685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.639734 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.639796 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:54:12.639968 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.640825 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.641595 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.641890 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.641950 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.642093 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:54:12.642147 (+ 54us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 08:54:12.642309 (+ 162us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41754: token expired
Metrics: {"server-negotiator.queue_time_us":79}
W20260619 08:54:12.642441 32298 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41754: token expired
W20260619 08:54:12.642477 32037 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.208.62:39305)
I20260619 08:54:12.648291 32298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.642762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41760 (local address 127.15.208.62:39305)
0619 08:54:12.642795 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.642797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.642832 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:54:12.642889 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.642890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.642897 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.642946 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:12.643402 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.644118 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.644863 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.645042 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.645095 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.646164 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:12.646165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:12.646165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:12.646171 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:12.647407 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.647712 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.647712 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.647713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.647761 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:12.647924 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:12.647924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:12.647925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:12.648031 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:12.648105 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.648181 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.648232 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:54:12.649526 32107 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:54:12.651136 32107 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:54:12.653520 32229 tablet_service.cc:1467] Processing CreateTablet for tablet 20a1be18431d4204bdad75e4dedfefde (DEFAULT_TABLE table=test-table [id=f5156c2521534f85bcb536be473277e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:12.653713 32229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a1be18431d4204bdad75e4dedfefde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:12.655040 32187 tablet_bootstrap.cc:492] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Bootstrap starting.
I20260619 08:54:12.655331 32187 tablet_bootstrap.cc:654] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:12.656160 32187 tablet_bootstrap.cc:492] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: No bootstrap required, opened a new log
I20260619 08:54:12.656213 32187 ts_tablet_manager.cc:1397] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:12.656361 32187 raft_consensus.cc:348] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.656435 32187 raft_consensus.cc:374] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:12.656481 32187 raft_consensus.cc:729] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bae253cd56147078fee97b395038889, State: Initialized, Role: FOLLOWER
I20260619 08:54:12.656529 32187 consensus_queue.cc:260] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.656615 32187 ts_tablet_manager.cc:1428] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:12.657029 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.653058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:43577 (local address 127.0.0.1:36620)
0619 08:54:12.653110 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:12.653121 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:12.653158 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:12.653538 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:12.653539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:12.653541 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:12.653586 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.653588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.654621 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.654622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.655646 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.655650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.656743 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.656745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.656815 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.656838 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 08:54:12.656904 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 08:54:12.656955 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:54:12.657089 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.653176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36620 (local address 127.15.208.2:43577)
0619 08:54:12.653354 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.653360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.653370 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:12.653428 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.653429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.653441 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.653537 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:12.653646 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.654548 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.655758 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.656607 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.656715 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.656798 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.656964 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.657006 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260619 08:54:12.657759 32367 tablet_service.cc:1467] Processing CreateTablet for tablet 20a1be18431d4204bdad75e4dedfefde (DEFAULT_TABLE table=test-table [id=f5156c2521534f85bcb536be473277e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:12.658084 32367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a1be18431d4204bdad75e4dedfefde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:12.658900 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.655084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37168 (local address 127.15.208.3:41589)
0619 08:54:12.655267 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.655272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.655282 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:12.655322 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.655324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.655339 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.655411 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:12.655611 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.656720 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.657963 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.658574 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.658653 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.658703 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.658791 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.658830 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
I20260619 08:54:12.658929 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.653105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:41589 (local address 127.0.0.1:37168)
0619 08:54:12.653273 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:12.653282 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:12.653323 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:12.655489 (+ 2166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:12.655490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:12.655492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:12.655532 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.655534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.656853 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.656854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.657867 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.657869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.658653 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.658654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.658714 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.658719 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:54:12.658776 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 08:54:12.658815 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":120,"threads_started":1}
I20260619 08:54:12.659663 32499 tablet_service.cc:1467] Processing CreateTablet for tablet 20a1be18431d4204bdad75e4dedfefde (DEFAULT_TABLE table=test-table [id=f5156c2521534f85bcb536be473277e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:12.660035 32499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a1be18431d4204bdad75e4dedfefde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:12.661625 32575 tablet_bootstrap.cc:492] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: Bootstrap starting.
I20260619 08:54:12.662171 32575 tablet_bootstrap.cc:654] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:12.662722 32575 log.cc:824] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:12.663117 32577 tablet_bootstrap.cc:492] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: Bootstrap starting.
I20260619 08:54:12.663550 32575 tablet_bootstrap.cc:492] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: No bootstrap required, opened a new log
I20260619 08:54:12.663605 32575 ts_tablet_manager.cc:1397] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:54:12.663757 32577 tablet_bootstrap.cc:654] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:12.664233 32577 log.cc:824] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:12.665062 32577 tablet_bootstrap.cc:492] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: No bootstrap required, opened a new log
I20260619 08:54:12.665138 32577 ts_tablet_manager.cc:1397] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:54:12.665410 32575 raft_consensus.cc:348] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.665588 32575 raft_consensus.cc:374] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:12.665624 32575 raft_consensus.cc:729] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 870fe4352c0c4c38bdeffeec60c8836a, State: Initialized, Role: FOLLOWER
I20260619 08:54:12.665745 32575 consensus_queue.cc:260] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.666024 32575 ts_tablet_manager.cc:1428] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:54:12.666044 32433 heartbeater.cc:503] Master 127.15.208.62:39305 was elected leader, sending a full tablet report...
I20260619 08:54:12.666795 32577 raft_consensus.cc:348] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.666920 32577 raft_consensus.cc:374] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:12.666989 32577 raft_consensus.cc:729] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93eb0670d25443678399215ef2336242, State: Initialized, Role: FOLLOWER
I20260619 08:54:12.667106 32577 consensus_queue.cc:260] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.667367 32577 ts_tablet_manager.cc:1428] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:54:12.667380 32565 heartbeater.cc:503] Master 127.15.208.62:39305 was elected leader, sending a full tablet report...
W20260619 08:54:12.734484 32434 tablet.cc:2369] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:54:12.837251 32296 tablet.cc:2369] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:54:12.866662 32566 tablet.cc:2369] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:54:12.894606 32301 raft_consensus.cc:482] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:12.894742 32301 raft_consensus.cc:504] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.895167 32301 leader_election.cc:288] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577), 93eb0670d25443678399215ef2336242 (127.15.208.3:41589)
I20260619 08:54:12.898219 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.895416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:56779 (local address 127.15.208.2:43577)
0619 08:54:12.895488 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.895490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.895539 (+ 49us) server_negotiation.cc:407] Connection header received
0619 08:54:12.895670 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.895671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.895682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.895758 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:12.895917 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.896423 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.897507 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.897872 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.897957 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.897965 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.898105 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.898144 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:54:12.898226 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.895534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:45237 (local address 127.15.208.3:41589)
0619 08:54:12.895591 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.895593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.895602 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:54:12.895693 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.895694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.895703 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.895779 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:12.895945 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.896437 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.897483 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.897855 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.897942 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.897951 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.898118 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.898156 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:54:12.898243 32582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.895413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:41589 (local address 127.15.208.1:45237)
0619 08:54:12.895550 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:12.895560 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:12.895619 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:12.895784 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:12.895785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:12.895788 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:12.895848 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.895850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.896535 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.896536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.897390 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.897393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.897943 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.897944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.898011 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.898031 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 08:54:12.898110 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:54:12.898154 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":2,"thread_start_us":57,"threads_started":1}
I20260619 08:54:12.898245 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.895339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:43577 (local address 127.15.208.1:56779)
0619 08:54:12.895469 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:12.895482 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:12.895567 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:12.895765 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:12.895766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:12.895768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:12.895828 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.895830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.896521 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.896522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.897401 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:12.897403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.897957 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.897958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:12.898030 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.898033 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:12.898099 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 08:54:12.898144 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":36,"thread_start_us":66,"threads_started":1}
I20260619 08:54:12.898574 32519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a1be18431d4204bdad75e4dedfefde" candidate_uuid: "3bae253cd56147078fee97b395038889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93eb0670d25443678399215ef2336242" is_pre_election: true
I20260619 08:54:12.898731 32519 raft_consensus.cc:2454] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bae253cd56147078fee97b395038889 in term 0.
I20260619 08:54:12.898701 32385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a1be18431d4204bdad75e4dedfefde" candidate_uuid: "3bae253cd56147078fee97b395038889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "870fe4352c0c4c38bdeffeec60c8836a" is_pre_election: true
I20260619 08:54:12.898860 32385 raft_consensus.cc:2454] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bae253cd56147078fee97b395038889 in term 0.
I20260619 08:54:12.898998 32181 leader_election.cc:302] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bae253cd56147078fee97b395038889, 93eb0670d25443678399215ef2336242; no voters:
I20260619 08:54:12.899152 32301 raft_consensus.cc:2781] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:54:12.899240 32301 raft_consensus.cc:482] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:54:12.899276 32301 raft_consensus.cc:3037] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:12.899999 32301 raft_consensus.cc:504] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.900164 32301 leader_election.cc:288] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 1 election: Requested vote from peers 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577), 93eb0670d25443678399215ef2336242 (127.15.208.3:41589)
I20260619 08:54:12.900331 32519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a1be18431d4204bdad75e4dedfefde" candidate_uuid: "3bae253cd56147078fee97b395038889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93eb0670d25443678399215ef2336242"
I20260619 08:54:12.900341 32385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a1be18431d4204bdad75e4dedfefde" candidate_uuid: "3bae253cd56147078fee97b395038889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "870fe4352c0c4c38bdeffeec60c8836a"
I20260619 08:54:12.900406 32385 raft_consensus.cc:3037] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:12.900429 32519 raft_consensus.cc:3037] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:12.901197 32385 raft_consensus.cc:2454] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bae253cd56147078fee97b395038889 in term 1.
I20260619 08:54:12.901222 32519 raft_consensus.cc:2454] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bae253cd56147078fee97b395038889 in term 1.
I20260619 08:54:12.901388 32182 leader_election.cc:302] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bae253cd56147078fee97b395038889, 870fe4352c0c4c38bdeffeec60c8836a; no voters:
I20260619 08:54:12.901499 32301 raft_consensus.cc:2781] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:12.901655 32301 raft_consensus.cc:686] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 LEADER]: Becoming Leader. State: Replica: 3bae253cd56147078fee97b395038889, State: Running, Role: LEADER
I20260619 08:54:12.901733 32301 consensus_queue.cc:237] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:12.902441 32107 catalog_manager.cc:5495] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bae253cd56147078fee97b395038889 (127.15.208.1). New cstate: current_term: 1 leader_uuid: "3bae253cd56147078fee97b395038889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } health_report { overall_health: HEALTHY } } }
I20260619 08:54:12.939843 32586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.937389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.15.208.1:40283)
0619 08:54:12.937561 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:12.937562 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:12.937569 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:54:12.937607 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:12.937608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:12.937627 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:12.937727 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:54:12.937863 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.938344 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.939097 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:12.939297 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:12.939357 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:12.939545 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:54:12.939647 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:54:12.939704 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:12.939747 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 08:54:12.939764 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
I20260619 08:54:12.941557 32519 raft_consensus.cc:1261] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242 [term 1 FOLLOWER]: Refusing update from remote peer 3bae253cd56147078fee97b395038889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:12.941574 32385 raft_consensus.cc:1261] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a [term 1 FOLLOWER]: Refusing update from remote peer 3bae253cd56147078fee97b395038889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:12.941865 32301 consensus_queue.cc:1035] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:12.941995 32583 consensus_queue.cc:1035] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:12.944226 32587 mvcc.cc:205] Tried to move back new op lower bound from 7298495500046106624 to 7298495499886030848. Current Snapshot: MvccSnapshot[applied={T|T < 7298495500046106624}]
I20260619 08:54:12.944711 32588 mvcc.cc:205] Tried to move back new op lower bound from 7298495500046106624 to 7298495499886030848. Current Snapshot: MvccSnapshot[applied={T|T < 7298495500046106624}]
I20260619 08:54:12.946750 32106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:12.946859 32106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:12.947575 32106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Sending DeleteTablet for 3 replicas of tablet 20a1be18431d4204bdad75e4dedfefde
I20260619 08:54:12.947916 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 32145
I20260619 08:54:12.947923 32499 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a1be18431d4204bdad75e4dedfefde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:12 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:37168
I20260619 08:54:12.948143 32367 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a1be18431d4204bdad75e4dedfefde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:12 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:36620
I20260619 08:54:12.948719 32596 tablet_replica.cc:330] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: stopping tablet replica
I20260619 08:54:12.948829 32596 raft_consensus.cc:2229] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:12.948944 32596 raft_consensus.cc:2258] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:12.949043 32597 tablet_replica.cc:330] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: stopping tablet replica
I20260619 08:54:12.949146 32597 raft_consensus.cc:2229] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:12.949239 32597 raft_consensus.cc:2258] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:12.949846 32596 ts_tablet_manager.cc:1905] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:12.950826 32597 ts_tablet_manager.cc:1905] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:12.951257 32596 ts_tablet_manager.cc:1918] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:12.951344 32596 log.cc:1196] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/20a1be18431d4204bdad75e4dedfefde
I20260619 08:54:12.951601 32596 ts_tablet_manager.cc:1939] T 20a1be18431d4204bdad75e4dedfefde P 93eb0670d25443678399215ef2336242: Deleting consensus metadata
I20260619 08:54:12.951995 32091 catalog_manager.cc:4841] TS 93eb0670d25443678399215ef2336242 (127.15.208.3:41589): tablet 20a1be18431d4204bdad75e4dedfefde (table test-table [id=f5156c2521534f85bcb536be473277e8]) successfully deleted
I20260619 08:54:12.952772 32597 ts_tablet_manager.cc:1918] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:12.952845 32597 log.cc:1196] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/20a1be18431d4204bdad75e4dedfefde
I20260619 08:54:12.953127 32597 ts_tablet_manager.cc:1939] T 20a1be18431d4204bdad75e4dedfefde P 870fe4352c0c4c38bdeffeec60c8836a: Deleting consensus metadata
I20260619 08:54:12.953557 32092 catalog_manager.cc:4841] TS 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577): tablet 20a1be18431d4204bdad75e4dedfefde (table test-table [id=f5156c2521534f85bcb536be473277e8]) successfully deleted
W20260619 08:54:12.953816 32092 connection.cc:535] client connection to 127.15.208.1:40283 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 08:54:12.953944 32092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:54:12.954291 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 32300
I20260619 08:54:12.954450 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:12.954299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:40283 (local address 127.0.0.1:49838)
0619 08:54:12.954371 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:12.954412 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:40283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:54:12.955027 32092 catalog_manager.cc:4584] TS 3bae253cd56147078fee97b395038889 (127.15.208.1:40283): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20a1be18431d4204bdad75e4dedfefde: Network error: Client connection negotiation failed: client connection to 127.15.208.1:40283: connect: Connection refused (error 111)
I20260619 08:54:12.959791 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 32436
I20260619 08:54:12.964357 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 32074
I20260619 08:54:32.970067 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39305
--webserver_interface=127.15.208.62
--webserver_port=32787
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:39305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:33.051854 32599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:33.052107 32599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:33.052145 32599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:33.053350 32599 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 08:54:33.053412 32599 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 08:54:33.053435 32599 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 08:54:33.053484 32599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:54:33.053508 32599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:33.053537 32599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:33.053565 32599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:54:33.053591 32599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:54:33.055065 32599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:39305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=32787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:33.055330 32599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:33.055567 32599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:33.060310 32604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:33.060295 32605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:33.060298 32607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:33.060623 32599 server_base.cc:1034] running on GCE node
I20260619 08:54:33.060896 32599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:33.061182 32599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:33.062358 32599 hybrid_clock.cc:648] HybridClock initialized: now 1781859273062354 us; error 35 us; skew 500 ppm
Jun 19 08:54:33 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859273, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:33.064827 32599 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:54:33.065303 32599 webserver.cc:460] Webserver started at http://127.15.208.62:32787/ using document root <none> and password file <none>
I20260619 08:54:33.065562 32599 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:33.065635 32599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:33.066926 32599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:33.067610 32614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:33.067788 32599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:33.067878 32599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "3521573a48134e558acf607515f56deb"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "6d63c4561cfcc4fdd1c7ce6bcce252a9"
server_key_iv: "08fdb93dfa7fffa7f9a6f16c98dc82cf"
server_key_version: "encryptionkey@0"
I20260619 08:54:33.068250 32599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:33.092314 32599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:33.095024 32599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:33.095212 32599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:33.099924 32599 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:39305
I20260619 08:54:33.099958 32666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:39305 every 8 connection(s)
I20260619 08:54:33.100385 32599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:54:33.101903 32667 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:54:33.102591 32667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Bootstrap starting.
I20260619 08:54:33.105655 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 32599
I20260619 08:54:33.106001 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:40283
--local_ip_for_outbound_sockets=127.15.208.1
--tserver_master_addrs=127.15.208.62:39305
--webserver_port=38683
--webserver_interface=127.15.208.1
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:54:33.111778 32667 log.cc:824] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:33.114233 32667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:54:33.114584 32667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Bootstrap complete.
I20260619 08:54:33.116956 32667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:54:33.117182 32667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3521573a48134e558acf607515f56deb, State: Initialized, Role: FOLLOWER
I20260619 08:54:33.117353 32667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:54:33.117409 32667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:54:33.117470 32667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:54:33.117514 32667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 2 FOLLOWER]: Advancing to term 3
I20260619 08:54:33.118492 32667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:54:33.118582 32667 leader_election.cc:302] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3521573a48134e558acf607515f56deb; no voters:
I20260619 08:54:33.118796 32667 leader_election.cc:288] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 08:54:33.118830 32671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 3 FOLLOWER]: Leader election won for term 3
I20260619 08:54:33.118945 32671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [term 3 LEADER]: Becoming Leader. State: Replica: 3521573a48134e558acf607515f56deb, State: Running, Role: LEADER
I20260619 08:54:33.119029 32671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } }
I20260619 08:54:33.119071 32667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:54:33.119341 32672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3521573a48134e558acf607515f56deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } } }
I20260619 08:54:33.119488 32672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:33.119882 32673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3521573a48134e558acf607515f56deb. Latest consensus state: current_term: 3 leader_uuid: "3521573a48134e558acf607515f56deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3521573a48134e558acf607515f56deb" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39305 } } }
I20260619 08:54:33.119976 32673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:33.120136 32675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:54:33.120946 32675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:54:33.121058 32675 catalog_manager.cc:1259] Loaded cluster ID: 6b2413d248de4759abf85340654d2686
I20260619 08:54:33.121094 32675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:54:33.122197 32675 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:54:33.122375 32675 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Loaded TSK: 0
I20260619 08:54:33.122423 32675 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3521573a48134e558acf607515f56deb: Loaded TSK: 1
F20260619 08:54:33.122643 32675 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f01823a39ad google::LogMessage::Fail() at ??:0
@ 0x7f01823a688a google::LogMessage::SendToLog() at ??:0
@ 0x7f01823a34de google::LogMessage::Flush() at ??:0
@ 0x7f01823a4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0184b10104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0184b0dd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f018a2fd1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f018a2fd48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f018a2fd594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f018a2c3c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f018a2cdad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f01842710d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0184266954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f01850ca6db start_thread at ??:0
@ 0x7f0180ea161f clone at ??:0
W20260619 08:54:33.217031 32669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:33.217314 32669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:33.217355 32669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:33.219038 32669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:33.219102 32669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:33.219142 32669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:54:33.221354 32669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:40283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=38683
--tserver_master_addrs=127.15.208.62:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:33.221626 32669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:33.221889 32669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:33.227074 32711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:33.227108 32709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:33.227108 32708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:33.227360 32669 server_base.cc:1034] running on GCE node
I20260619 08:54:33.227730 32669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:33.228006 32669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:33.229195 32669 hybrid_clock.cc:648] HybridClock initialized: now 1781859273229170 us; error 28 us; skew 500 ppm
Jun 19 08:54:33 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859273, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:33.232213 32669 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:54:33.232761 32669 webserver.cc:460] Webserver started at http://127.15.208.1:38683/ using document root <none> and password file <none>
I20260619 08:54:33.233029 32669 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:33.233099 32669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:33.234879 32669 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:54:33.235680 32718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:33.235883 32669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:33.235977 32669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "3bae253cd56147078fee97b395038889"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "d5a8ff8c3aae0876a12c8adfb132c844"
server_key_iv: "e5d9189b5567f4434900190425dae64e"
server_key_version: "encryptionkey@0"
I20260619 08:54:33.236426 32669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:33.253541 32669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:33.255995 32669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:33.256165 32669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:33.256417 32669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:33.256960 32725 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:54:33.258389 32669 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:54:33.258450 32669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:54:33.258486 32669 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:54:33.259341 32669 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:54:33.259408 32669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:54:33.259485 32725 tablet_bootstrap.cc:492] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Bootstrap starting.
I20260619 08:54:33.266992 32669 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:40283
I20260619 08:54:33.267212 364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:40283 every 8 connection(s)
I20260619 08:54:33.267477 32669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:54:33.270392 32725 log.cc:824] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:33.271867 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 32669
I20260619 08:54:33.272190 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:43577
--local_ip_for_outbound_sockets=127.15.208.2
--tserver_master_addrs=127.15.208.62:39305
--webserver_port=38247
--webserver_interface=127.15.208.2
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:54:33.272958 32725 tablet_bootstrap.cc:492] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:54:33.273265 32725 tablet_bootstrap.cc:492] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Bootstrap complete.
I20260619 08:54:33.274232 32725 ts_tablet_manager.cc:1397] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260619 08:54:33.275241 32725 raft_consensus.cc:348] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:33.275487 32725 raft_consensus.cc:729] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bae253cd56147078fee97b395038889, State: Initialized, Role: FOLLOWER
I20260619 08:54:33.275658 32725 consensus_queue.cc:260] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:33.275943 32725 ts_tablet_manager.cc:1428] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260619 08:54:33.343528 367 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:54:33.268209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:59113)
0619 08:54:33.268429 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:33.268480 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:33.269281 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:33.343311 (+ 74030us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":112,"threads_started":1}
W20260619 08:54:33.343796 365 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:39305 (0 consecutive failures): Network error: Failed to ping master at 127.15.208.62:39305: Client connection negotiation failed: client connection to 127.15.208.62:39305: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:54:33.344135 367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:33.344014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:54565)
0619 08:54:33.344070 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:33.344106 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:54:33.344537 367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:33.344450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:58979)
0619 08:54:33.344493 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:33.344519 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 08:54:33.344667 365 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:54:33.356271 369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:33.356482 369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:33.356505 369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:33.357667 369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:33.357712 369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:33.357738 369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:54:33.359232 369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:43577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=38247
--tserver_master_addrs=127.15.208.62:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:33.359427 369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:33.359601 369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:33.364539 375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:33.364880 378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:33.364997 376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:33.365172 369 server_base.cc:1034] running on GCE node
I20260619 08:54:33.365375 369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:33.365592 369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:33.366781 369 hybrid_clock.cc:648] HybridClock initialized: now 1781859273366765 us; error 34 us; skew 500 ppm
Jun 19 08:54:33 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859273, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:33.369366 369 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:54:33.369823 369 webserver.cc:460] Webserver started at http://127.15.208.2:38247/ using document root <none> and password file <none>
I20260619 08:54:33.370059 369 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:33.370105 369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:33.371469 369 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:33.372083 385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:33.372268 369 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:33.372339 369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "870fe4352c0c4c38bdeffeec60c8836a"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "c44e2fb1cef99dfa4c391f0514bdfbb5"
server_key_iv: "3db30fb4ff1b9e114b861af2de4018ba"
server_key_version: "encryptionkey@0"
I20260619 08:54:33.372692 369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:33.395191 369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:33.397794 369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:33.397977 369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:33.398236 369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:33.398571 369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:33.398630 369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:33.398669 369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:33.398697 369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:33.404718 369 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:43577
I20260619 08:54:33.404819 498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:43577 every 8 connection(s)
I20260619 08:54:33.405151 369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:54:33.406229 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:33.405827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:37745)
0619 08:54:33.405979 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:33.406076 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":123,"threads_started":1}
W20260619 08:54:33.406672 499 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:39305 (0 consecutive failures): Network error: Failed to ping master at 127.15.208.62:39305: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
I20260619 08:54:33.406970 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:33.406873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:43485)
0619 08:54:33.406924 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:33.406952 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:54:33.407249 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:33.407176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:54189)
0619 08:54:33.407215 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:33.407236 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 08:54:33.407366 499 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:54:33.408928 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 369
I20260619 08:54:33.409224 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:41589
--local_ip_for_outbound_sockets=127.15.208.3
--tserver_master_addrs=127.15.208.62:39305
--webserver_port=39571
--webserver_interface=127.15.208.3
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:33.492017 502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:33.492220 502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:33.492256 502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:33.493502 502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:33.493582 502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:33.493626 502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:54:33.495098 502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:41589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=39571
--tserver_master_addrs=127.15.208.62:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:33.495362 502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:33.495610 502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:33.501127 508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:33.501173 507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:33.501438 510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:33.501569 502 server_base.cc:1034] running on GCE node
I20260619 08:54:33.501796 502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:33.502101 502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:33.503266 502 hybrid_clock.cc:648] HybridClock initialized: now 1781859273503245 us; error 43 us; skew 500 ppm
Jun 19 08:54:33 dist-test-slave-p0h9 krb5kdc[31527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859273, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:33.505947 502 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:54:33.506404 502 webserver.cc:460] Webserver started at http://127.15.208.3:39571/ using document root <none> and password file <none>
I20260619 08:54:33.506671 502 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:33.506745 502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:33.508102 502 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:33.508677 517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:33.508881 502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:54:33.508988 502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "93eb0670d25443678399215ef2336242"
format_stamp: "Formatted at 2026-06-19 08:53:51 on dist-test-slave-p0h9"
server_key: "a9f1c94b6d99923917f7a0a4947b932c"
server_key_iv: "87bb0a24b8ed55f2527dfc1e06dbec7f"
server_key_version: "encryptionkey@0"
I20260619 08:54:33.509341 502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:33.520010 502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:33.521951 502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:33.522122 502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:33.522392 502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:33.522706 502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:33.522761 502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:33.522804 502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:33.522828 502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:33.528916 502 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:41589
I20260619 08:54:33.529032 630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:41589 every 8 connection(s)
I20260619 08:54:33.529332 502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:54:33.530423 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:33.530046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:58173)
0619 08:54:33.530217 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:33.530287 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
W20260619 08:54:33.530846 631 heartbeater.cc:644] Failed to heartbeat to 127.15.208.62:39305 (0 consecutive failures): Network error: Failed to ping master at 127.15.208.62:39305: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
I20260619 08:54:33.531108 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:33.531021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:33851)
0619 08:54:33.531063 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:33.531090 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 08:54:33.531374 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:33.531303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:40759)
0619 08:54:33.531341 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:33.531361 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 08:54:33.531495 631 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:54:33.534204 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 502
I20260619 08:54:34.345278 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:34.345022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:46443)
0619 08:54:34.345152 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:34.345214 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260619 08:54:34.407924 636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:34.407704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:46495)
0619 08:54:34.407831 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:34.407871 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260619 08:54:34.532114 637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:34.531868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:45203)
0619 08:54:34.532011 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:34.532055 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260619 08:54:34.741940 638 raft_consensus.cc:482] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:34.742105 638 raft_consensus.cc:504] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:34.742491 638 leader_election.cc:288] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577), 93eb0670d25443678399215ef2336242 (127.15.208.3:41589)
Jun 19 08:54:34 dist-test-slave-p0h9 krb5kdc[31527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859273, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.2@KRBTEST.COM
Jun 19 08:54:34 dist-test-slave-p0h9 krb5kdc[31527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859273, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.3@KRBTEST.COM
I20260619 08:54:34.753741 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:34.742902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:58675 (local address 127.15.208.2:43577)
0619 08:54:34.743050 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:34.743063 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:34.743105 (+ 42us) server_negotiation.cc:407] Connection header received
0619 08:54:34.743563 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:34.743565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:34.743605 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:34.743769 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:34.744501 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:34.745445 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:34.746404 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:34.746825 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:34.746942 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:34.749378 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:34.749382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:34.749383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:34.749450 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:34.751507 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:34.752044 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:34.752047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:34.752047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:34.752179 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:34.752611 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:34.752612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:34.752613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:34.752991 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:34.753083 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:34.753488 (+ 405us) server_negotiation.cc:299] Negotiation successful
0619 08:54:34.753620 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260619 08:54:34.753742 641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:34.742922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:45563 (local address 127.15.208.3:41589)
0619 08:54:34.743099 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:34.743107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:34.743144 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:54:34.743561 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:34.743564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:34.743605 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:34.743769 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:34.744531 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:34.745444 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:34.746426 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:34.746824 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:34.746944 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:34.749827 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:34.749831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:34.749831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:34.749843 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:34.751748 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:34.752131 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:34.752133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:34.752134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:34.752236 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:34.752611 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:34.752612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:34.752613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:34.752991 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:34.753083 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:34.753476 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 08:54:34.753620 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
I20260619 08:54:34.753753 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:34.742680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:41589 (local address 127.15.208.1:45563)
0619 08:54:34.742748 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:34.742760 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:34.742818 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:34.743802 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:34.743804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:34.744194 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:34.744373 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:34.744379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:34.745543 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:34.745544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:34.746305 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:34.746307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:34.746923 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:34.746924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:34.747037 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:34.747599 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:34.747608 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:34.749652 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:34.751834 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:34.751836 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:34.751836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:34.752052 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:34.752329 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:34.752330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:34.752330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:34.752509 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:34.753076 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:34.753085 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:34.753268 (+ 183us) client_negotiation.cc:769] Sending connection context
0619 08:54:34.753393 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 08:54:34.753618 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":78}
I20260619 08:54:34.754184 639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:34.742762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:43577 (local address 127.15.208.1:58675)
0619 08:54:34.742885 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:34.742896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:34.742966 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:34.743802 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:34.743804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:34.744195 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:34.744374 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:34.744379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:34.745562 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:34.745563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:34.746304 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:34.746305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:34.746925 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:34.746926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:34.747037 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:34.747653 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:34.747657 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:34.749263 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:34.751599 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:34.751606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:34.751607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:34.751949 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:34.752329 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:34.752330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:34.752330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:34.752505 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:34.753076 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:34.753085 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:34.753268 (+ 183us) client_negotiation.cc:769] Sending connection context
0619 08:54:34.753393 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 08:54:34.753673 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":148,"thread_start_us":66,"threads_started":1}
W20260619 08:54:34.754686 32719 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93eb0670d25443678399215ef2336242 (127.15.208.3:41589): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
W20260619 08:54:34.754988 32720 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
I20260619 08:54:34.755054 32720 leader_election.cc:302] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bae253cd56147078fee97b395038889; no voters: 870fe4352c0c4c38bdeffeec60c8836a, 93eb0670d25443678399215ef2336242
I20260619 08:54:34.755157 638 raft_consensus.cc:2726] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 08:54:35.346006 642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:35.345800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:42139)
0619 08:54:35.345926 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:35.345962 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260619 08:54:35.408653 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:35.408430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:44607)
0619 08:54:35.408563 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:35.408603 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260619 08:54:35.533107 644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:35.532903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:57843)
0619 08:54:35.533035 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:35.533069 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260619 08:54:36.346652 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:36.346427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:56835)
0619 08:54:36.346574 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:36.346604 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260619 08:54:36.409315 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:36.409123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:49163)
0619 08:54:36.409240 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:36.409274 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260619 08:54:36.424196 647 raft_consensus.cc:482] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:36.424341 647 raft_consensus.cc:504] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:36.424621 647 leader_election.cc:288] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577), 93eb0670d25443678399215ef2336242 (127.15.208.3:41589)
W20260619 08:54:36.425112 32720 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
W20260619 08:54:36.425248 32719 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93eb0670d25443678399215ef2336242 (127.15.208.3:41589): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
I20260619 08:54:36.425325 32719 leader_election.cc:302] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bae253cd56147078fee97b395038889; no voters: 870fe4352c0c4c38bdeffeec60c8836a, 93eb0670d25443678399215ef2336242
I20260619 08:54:36.425442 647 raft_consensus.cc:2726] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 08:54:36.533841 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:36.533588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:37465)
0619 08:54:36.533749 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:36.533795 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260619 08:54:37.347354 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:37.347148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:34585)
0619 08:54:37.347272 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:37.347311 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260619 08:54:37.409983 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:37.409731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:43609)
0619 08:54:37.409895 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:37.409938 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":61,"threads_started":1}
I20260619 08:54:37.534533 651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:37.534326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:39307)
0619 08:54:37.534443 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:37.534474 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260619 08:54:38.348093 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:38.347867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:48705)
0619 08:54:38.347996 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:38.348044 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260619 08:54:38.410666 653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:38.410472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:60519)
0619 08:54:38.410589 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:38.410623 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260619 08:54:38.460332 654 raft_consensus.cc:482] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:38.460466 654 raft_consensus.cc:504] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:38.460697 654 leader_election.cc:288] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577), 93eb0670d25443678399215ef2336242 (127.15.208.3:41589)
W20260619 08:54:38.461086 32719 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93eb0670d25443678399215ef2336242 (127.15.208.3:41589): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
W20260619 08:54:38.461164 32720 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
I20260619 08:54:38.461222 32720 leader_election.cc:302] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bae253cd56147078fee97b395038889; no voters: 870fe4352c0c4c38bdeffeec60c8836a, 93eb0670d25443678399215ef2336242
I20260619 08:54:38.461318 654 raft_consensus.cc:2726] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 08:54:38.535228 655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:38.534974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:48623)
0619 08:54:38.535123 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:38.535166 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260619 08:54:39.348896 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:39.348591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:50879)
0619 08:54:39.348813 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:39.348851 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":60,"threads_started":1}
I20260619 08:54:39.411310 657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:39.411073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:50673)
0619 08:54:39.411203 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:39.411237 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260619 08:54:39.535951 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:39.535720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:53159)
0619 08:54:39.535863 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:39.535905 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260619 08:54:40.349617 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:40.349392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:33617)
0619 08:54:40.349537 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:40.349571 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":62,"threads_started":1}
I20260619 08:54:40.367107 660 raft_consensus.cc:482] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:40.367208 660 raft_consensus.cc:504] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:40.367377 660 leader_election.cc:288] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577), 93eb0670d25443678399215ef2336242 (127.15.208.3:41589)
W20260619 08:54:40.367754 32720 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
W20260619 08:54:40.367875 32719 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93eb0670d25443678399215ef2336242 (127.15.208.3:41589): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
I20260619 08:54:40.367933 32719 leader_election.cc:302] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bae253cd56147078fee97b395038889; no voters: 870fe4352c0c4c38bdeffeec60c8836a, 93eb0670d25443678399215ef2336242
I20260619 08:54:40.368013 660 raft_consensus.cc:2726] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 08:54:40.412110 661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:40.411817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:56377)
0619 08:54:40.412027 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:40.412067 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":62,"threads_started":1}
I20260619 08:54:40.536681 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:40.536438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:42115)
0619 08:54:40.536574 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:40.536615 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260619 08:54:41.350293 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:41.350050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:51979)
0619 08:54:41.350194 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:41.350246 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260619 08:54:41.412894 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:41.412580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:53911)
0619 08:54:41.412784 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:41.412833 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":61,"threads_started":1}
I20260619 08:54:41.537456 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:41.537221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:50695)
0619 08:54:41.537358 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:41.537404 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260619 08:54:42.351007 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:42.350783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:54043)
0619 08:54:42.350920 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:42.350963 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260619 08:54:42.413585 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:42.413376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:60465)
0619 08:54:42.413498 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:42.413542 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260619 08:54:42.538141 668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:42.537940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:54201)
0619 08:54:42.538057 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:42.538093 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260619 08:54:43.351792 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:43.351582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:51505)
0619 08:54:43.351706 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:43.351744 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260619 08:54:43.414255 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:43.414028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:54387)
0619 08:54:43.414172 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:43.414209 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260619 08:54:43.476852 671 raft_consensus.cc:482] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:43.476958 671 raft_consensus.cc:504] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:43.477238 671 leader_election.cc:288] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577), 93eb0670d25443678399215ef2336242 (127.15.208.3:41589)
W20260619 08:54:43.477699 32720 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
W20260619 08:54:43.477749 32719 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93eb0670d25443678399215ef2336242 (127.15.208.3:41589): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
I20260619 08:54:43.477777 32720 leader_election.cc:302] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bae253cd56147078fee97b395038889; no voters: 870fe4352c0c4c38bdeffeec60c8836a, 93eb0670d25443678399215ef2336242
I20260619 08:54:43.477843 671 raft_consensus.cc:2726] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 08:54:43.538928 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:43.538645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:38165)
0619 08:54:43.538825 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:43.538873 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
I20260619 08:54:44.352463 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:44.352269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:37001)
0619 08:54:44.352387 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:44.352418 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260619 08:54:44.415129 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:44.414844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:43919)
0619 08:54:44.415014 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:44.415068 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260619 08:54:44.539988 675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:44.539708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:39099)
0619 08:54:44.539885 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:44.539939 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":116,"threads_started":1}
I20260619 08:54:45.225898 676 raft_consensus.cc:482] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:45.226058 676 raft_consensus.cc:504] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870fe4352c0c4c38bdeffeec60c8836a" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 43577 } } peers { permanent_uuid: "93eb0670d25443678399215ef2336242" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 41589 } } peers { permanent_uuid: "3bae253cd56147078fee97b395038889" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 40283 } }
I20260619 08:54:45.226290 676 leader_election.cc:288] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577), 93eb0670d25443678399215ef2336242 (127.15.208.3:41589)
W20260619 08:54:45.226900 32719 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93eb0670d25443678399215ef2336242 (127.15.208.3:41589): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
W20260619 08:54:45.226989 32720 leader_election.cc:341] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870fe4352c0c4c38bdeffeec60c8836a (127.15.208.2:43577): Not found: Tablet not found: 20a1be18431d4204bdad75e4dedfefde
I20260619 08:54:45.227036 32720 leader_election.cc:302] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bae253cd56147078fee97b395038889; no voters: 870fe4352c0c4c38bdeffeec60c8836a, 93eb0670d25443678399215ef2336242
I20260619 08:54:45.227118 676 raft_consensus.cc:2726] T 20a1be18431d4204bdad75e4dedfefde P 3bae253cd56147078fee97b395038889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 08:54:45.353127 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:45.352919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:51897)
0619 08:54:45.353046 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:45.353078 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260619 08:54:45.415985 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:45.415720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:51119)
0619 08:54:45.415877 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:45.415938 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":61,"threads_started":1}
I20260619 08:54:45.540812 679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:45.540508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:43417)
0619 08:54:45.540677 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:45.540719 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260619 08:54:46.353830 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:46.353622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:60657)
0619 08:54:46.353744 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:46.353784 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260619 08:54:46.416685 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:46.416440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:56105)
0619 08:54:46.416599 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:46.416640 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":68,"threads_started":1}
I20260619 08:54:46.541438 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:46.541222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:57117)
0619 08:54:46.541357 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:46.541394 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260619 08:54:47.354590 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:47.354344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:54041)
0619 08:54:47.354499 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:47.354544 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260619 08:54:47.417526 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:47.417274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:45321)
0619 08:54:47.417419 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:47.417468 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260619 08:54:47.542102 685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:47.541879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:36615)
0619 08:54:47.542012 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:47.542058 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260619 08:54:48.355397 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:48.355176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.1:41741)
0619 08:54:48.355312 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:48.355354 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260619 08:54:48.418292 687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:48.418075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.2:43525)
0619 08:54:48.418207 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:48.418247 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":60,"threads_started":1}
I20260619 08:54:48.535197 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 32669
I20260619 08:54:48.539953 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 369
I20260619 08:54:48.542989 688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:48.542782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39305 (local address 127.15.208.3:57139)
0619 08:54:48.542917 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:48.542949 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.62:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260619 08:54:48.544394 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 502
2026-06-19T08:54:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58630 ms)
[----------] 30 tests from SecurityITest (230801 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:54:48 dist-test-slave-p0h9 krb5kdc[691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:54:48 dist-test-slave-p0h9 krb5kdc[691](info): set up 2 sockets
Jun 19 08:54:48 dist-test-slave-p0h9 krb5kdc[691](info): commencing operation
krb5kdc: starting...
W20260619 08:54:49.581065 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:54:49 dist-test-slave-p0h9 krb5kdc[691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:54:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:54:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:54:49.697726 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:38817
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:38817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:49.778090 704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:49.778294 704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:49.778317 704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:49.779487 704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:54:49.779531 704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:49.779544 704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:49.779557 704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:54:49.779569 704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:54:49.781100 704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:38817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:38817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:49.781358 704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:49.781612 704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:49.786993 710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:49.787055 704 server_base.cc:1034] running on GCE node
W20260619 08:54:49.787071 709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:49.787029 712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:49.787650 704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:49.787959 704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:49.789141 704 hybrid_clock.cc:648] HybridClock initialized: now 1781859289789113 us; error 41 us; skew 500 ppm
Jun 19 08:54:49 dist-test-slave-p0h9 krb5kdc[691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859289, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:49.791944 704 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:54:49.792495 704 webserver.cc:460] Webserver started at http://127.15.208.62:34739/ using document root <none> and password file <none>
I20260619 08:54:49.792796 704 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:49.792874 704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:49.793058 704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:49.794234 704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "564a9cf250bf4af1a4040aa9183da9bc"
format_stamp: "Formatted at 2026-06-19 08:54:49 on dist-test-slave-p0h9"
server_key: "52a1f2415ef71631e910da3290950068"
server_key_iv: "131343904feac3120640e30d4803c4e5"
server_key_version: "encryptionkey@0"
I20260619 08:54:49.794644 704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "564a9cf250bf4af1a4040aa9183da9bc"
format_stamp: "Formatted at 2026-06-19 08:54:49 on dist-test-slave-p0h9"
server_key: "52a1f2415ef71631e910da3290950068"
server_key_iv: "131343904feac3120640e30d4803c4e5"
server_key_version: "encryptionkey@0"
I20260619 08:54:49.796126 704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:54:49.796934 719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:49.797124 704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:54:49.797241 704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "564a9cf250bf4af1a4040aa9183da9bc"
format_stamp: "Formatted at 2026-06-19 08:54:49 on dist-test-slave-p0h9"
server_key: "52a1f2415ef71631e910da3290950068"
server_key_iv: "131343904feac3120640e30d4803c4e5"
server_key_version: "encryptionkey@0"
I20260619 08:54:49.797322 704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:49.816288 704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:49.818936 704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:49.819121 704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:49.823804 704 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:38817
I20260619 08:54:49.823840 771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:38817 every 8 connection(s)
I20260619 08:54:49.824220 704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:54:49.825160 772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:49.827382 772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc: Bootstrap starting.
I20260619 08:54:49.827937 772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:49.828166 772 log.cc:824] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:49.828915 772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc: No bootstrap required, opened a new log
I20260619 08:54:49.830443 772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564a9cf250bf4af1a4040aa9183da9bc" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38817 } }
I20260619 08:54:49.830585 772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:49.830606 772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564a9cf250bf4af1a4040aa9183da9bc, State: Initialized, Role: FOLLOWER
I20260619 08:54:49.830746 772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564a9cf250bf4af1a4040aa9183da9bc" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38817 } }
I20260619 08:54:49.830830 772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:54:49.830853 772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:54:49.830883 772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:49.831359 772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564a9cf250bf4af1a4040aa9183da9bc" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38817 } }
I20260619 08:54:49.831440 772 leader_election.cc:302] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 564a9cf250bf4af1a4040aa9183da9bc; no voters:
I20260619 08:54:49.831570 772 leader_election.cc:288] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:54:49.831645 775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:49.831840 775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [term 1 LEADER]: Becoming Leader. State: Replica: 564a9cf250bf4af1a4040aa9183da9bc, State: Running, Role: LEADER
I20260619 08:54:49.831837 772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:54:49.831987 775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564a9cf250bf4af1a4040aa9183da9bc" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38817 } }
I20260619 08:54:49.832533 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 704
I20260619 08:54:49.832653 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:54:49.832747 777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 564a9cf250bf4af1a4040aa9183da9bc. Latest consensus state: current_term: 1 leader_uuid: "564a9cf250bf4af1a4040aa9183da9bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564a9cf250bf4af1a4040aa9183da9bc" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38817 } } }
I20260619 08:54:49.832832 777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:49.832943 16192 external_mini_cluster.cc:1428] Setting key 788bd86b74dd3c1bc33af018babf2a42
I20260619 08:54:49.833235 776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "564a9cf250bf4af1a4040aa9183da9bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564a9cf250bf4af1a4040aa9183da9bc" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38817 } } }
I20260619 08:54:49.833321 785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:54:49.833442 776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:49.834008 785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:54:49.835681 785 catalog_manager.cc:1347] Generated new cluster ID: 7de47a3693aa4894b2809e1d0b2bdd8e
I20260619 08:54:49.835734 785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:54:49 dist-test-slave-p0h9 krb5kdc[691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:49.844188 793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:49.833653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.15.208.62:38817)
0619 08:54:49.833872 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:49.833881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:49.833948 (+ 67us) server_negotiation.cc:407] Connection header received
0619 08:54:49.834051 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:49.834056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:49.834157 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:49.834329 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:49.835176 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:49.836408 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:49.837412 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:49.837893 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:49.838045 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:49.840559 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:49.840563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:49.840563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:49.840576 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:49.842475 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:49.842885 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:49.842888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:49.842889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:49.843001 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:49.843257 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:49.843258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:49.843259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:49.843527 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:49.843728 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:49.843803 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 08:54:49.843962 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":77,"threads_started":1}
I20260619 08:54:49.862550 785 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:54:49.863379 785 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:54:49.871591 785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc: Generated new TSK 0
I20260619 08:54:49.871793 785 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:54:49.904227 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38817
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:49.985381 798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:49.985590 798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:49.985615 798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:49.986747 798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:49.986788 798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:49.986842 798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:54:49.988268 798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:49.988515 798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:49.988785 798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:49.993943 803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:49.993961 806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:49.994078 798 server_base.cc:1034] running on GCE node
W20260619 08:54:49.994194 804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:49.994599 798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:49.994908 798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:49.996066 798 hybrid_clock.cc:648] HybridClock initialized: now 1781859289996051 us; error 34 us; skew 500 ppm
Jun 19 08:54:49 dist-test-slave-p0h9 krb5kdc[691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859289, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:49.998569 798 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:54:49.999051 798 webserver.cc:460] Webserver started at http://127.15.208.1:38149/ using document root <none> and password file <none>
I20260619 08:54:49.999275 798 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:49.999341 798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:49.999500 798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:50.000609 798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "2145f8e91a0c47edaa3b18685dc9f972"
format_stamp: "Formatted at 2026-06-19 08:54:49 on dist-test-slave-p0h9"
server_key: "fb96e57d51740e28eedda985e37c7992"
server_key_iv: "107ded21d09c9a0eee51991674b8d67f"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.001015 798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "2145f8e91a0c47edaa3b18685dc9f972"
format_stamp: "Formatted at 2026-06-19 08:54:49 on dist-test-slave-p0h9"
server_key: "fb96e57d51740e28eedda985e37c7992"
server_key_iv: "107ded21d09c9a0eee51991674b8d67f"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.002398 798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:50.003194 813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.003410 798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:50.003515 798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "2145f8e91a0c47edaa3b18685dc9f972"
format_stamp: "Formatted at 2026-06-19 08:54:49 on dist-test-slave-p0h9"
server_key: "fb96e57d51740e28eedda985e37c7992"
server_key_iv: "107ded21d09c9a0eee51991674b8d67f"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.003607 798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:50.025331 798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:50.027307 798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:50.027482 798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:50.027745 798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:50.028067 798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:50.028124 798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.028167 798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:50.028213 798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.034371 798 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:44901
I20260619 08:54:50.034454 926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:44901 every 8 connection(s)
I20260619 08:54:50.034785 798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:54:50.039462 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 798
I20260619 08:54:50.039590 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:54:50.039765 16192 external_mini_cluster.cc:1428] Setting key d1bccf577b5e2402c4f783afc95653b8
Jun 19 08:54:50 dist-test-slave-p0h9 krb5kdc[691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859289, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:50.044605 793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.035583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:49547 (local address 127.15.208.62:38817)
0619 08:54:50.035653 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.035654 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.035871 (+ 217us) server_negotiation.cc:407] Connection header received
0619 08:54:50.036352 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.036353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.036366 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.036425 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:50.037380 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.037883 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.038844 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.039032 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.039084 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.041455 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:50.041457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:50.041457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:50.041466 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:50.043118 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:50.043502 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:50.043503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:50.043503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:50.043575 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:50.043805 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:50.043806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:50.043806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:50.043959 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:50.044035 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.044379 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.044482 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:54:50.044607 929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.035551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38817 (local address 127.15.208.1:49547)
0619 08:54:50.035743 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.035786 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.036271 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.036495 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.036498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.037110 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:50.037303 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.037307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.037963 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.037964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.038770 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.038771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.039120 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.039121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.039218 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.039787 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:50.039803 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:50.041366 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:50.043212 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:50.043218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:50.043219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:50.043427 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:50.043641 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:50.043642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:50.043642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:50.043738 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:50.044062 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:50.044069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:50.044237 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.044327 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.044446 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":121,"threads_started":1}
I20260619 08:54:50.045158 927 heartbeater.cc:347] Connected to a master server at 127.15.208.62:38817
I20260619 08:54:50.045280 927 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:50.045467 927 heartbeater.cc:511] Master 127.15.208.62:38817 requested a full tablet report, sending...
I20260619 08:54:50.046044 736 ts_manager.cc:194] Registered new tserver with Master: 2145f8e91a0c47edaa3b18685dc9f972 (127.15.208.1:44901)
I20260619 08:54:50.046968 736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:49547
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:54:50.063633 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38817
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:50.144989 932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:50.145200 932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:50.145221 932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:50.146276 932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:50.146309 932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:50.146335 932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:54:50.147677 932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:50.147899 932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:50.148115 932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:50.153236 940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:50.153227 938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:50.153227 937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:50.153472 932 server_base.cc:1034] running on GCE node
I20260619 08:54:50.153764 932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:50.154013 932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:50.155154 932 hybrid_clock.cc:648] HybridClock initialized: now 1781859290155138 us; error 41 us; skew 500 ppm
Jun 19 08:54:50 dist-test-slave-p0h9 krb5kdc[691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859290, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:50.157819 932 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:54:50.158311 932 webserver.cc:460] Webserver started at http://127.15.208.2:36391/ using document root <none> and password file <none>
I20260619 08:54:50.158533 932 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:50.158605 932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:50.158757 932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:50.159821 932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "58bba4d5ae334c45840b819bb355752b"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "4b314b3fbcecdadd1a7ce038a0f50f4e"
server_key_iv: "2755aecbaae55fa02480cb9c31ec907a"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.160192 932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "58bba4d5ae334c45840b819bb355752b"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "4b314b3fbcecdadd1a7ce038a0f50f4e"
server_key_iv: "2755aecbaae55fa02480cb9c31ec907a"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.161656 932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:54:50.162441 947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.162663 932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:54:50.162770 932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "58bba4d5ae334c45840b819bb355752b"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "4b314b3fbcecdadd1a7ce038a0f50f4e"
server_key_iv: "2755aecbaae55fa02480cb9c31ec907a"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.162845 932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:50.183949 932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:50.185964 932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:50.186153 932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:50.186405 932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:50.186709 932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:50.186762 932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.186806 932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:50.186837 932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.193284 932 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:34391
I20260619 08:54:50.193372 1060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:34391 every 8 connection(s)
I20260619 08:54:50.193677 932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:54:50.198727 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 932
I20260619 08:54:50.198859 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:54:50.199074 16192 external_mini_cluster.cc:1428] Setting key 611b611596c6f0f73056ca128adf2564
Jun 19 08:54:50 dist-test-slave-p0h9 krb5kdc[691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859290, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:50.203857 793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.194492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:39291 (local address 127.15.208.62:38817)
0619 08:54:50.194559 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.194561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.194804 (+ 243us) server_negotiation.cc:407] Connection header received
0619 08:54:50.195257 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.195258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.195271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.195336 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:50.196323 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.196940 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.197897 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.198097 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.198163 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.200628 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:50.200630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:50.200630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:50.200639 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:50.202284 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:50.202692 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:50.202694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:50.202694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:50.202760 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:50.203071 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:50.203072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:50.203073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:50.203238 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:50.203315 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.203642 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.203739 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:54:50.203886 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.194493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38817 (local address 127.15.208.2:39291)
0619 08:54:50.194668 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.194703 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.195169 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.195390 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.195393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.196035 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:50.196234 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.196239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.197028 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.197028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.197820 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.197822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.198183 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.198184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.198282 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.198774 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:50.198784 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:50.200519 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:50.202365 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:50.202371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:50.202373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:50.202617 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:50.202858 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:50.202859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:50.202860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:50.202972 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:50.203335 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:50.203342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:50.203504 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.203594 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.203722 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260619 08:54:50.204468 1061 heartbeater.cc:347] Connected to a master server at 127.15.208.62:38817
I20260619 08:54:50.204588 1061 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:50.204842 1061 heartbeater.cc:511] Master 127.15.208.62:38817 requested a full tablet report, sending...
I20260619 08:54:50.205288 736 ts_manager.cc:194] Registered new tserver with Master: 58bba4d5ae334c45840b819bb355752b (127.15.208.2:34391)
I20260619 08:54:50.205675 736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:39291
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:54:50.223807 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38817
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:50.322228 1066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:50.322480 1066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:50.322520 1066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:50.323688 1066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:50.323764 1066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:50.323810 1066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:54:50.325353 1066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:38817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:50.325676 1066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:50.325922 1066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:50.331282 1074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:50.331339 1066 server_base.cc:1034] running on GCE node
W20260619 08:54:50.331290 1072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:50.331287 1071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:50.331818 1066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:50.332074 1066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:50.333235 1066 hybrid_clock.cc:648] HybridClock initialized: now 1781859290333227 us; error 39 us; skew 500 ppm
Jun 19 08:54:50 dist-test-slave-p0h9 krb5kdc[691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859290, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:50.336122 1066 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:54:50.336617 1066 webserver.cc:460] Webserver started at http://127.15.208.3:38613/ using document root <none> and password file <none>
I20260619 08:54:50.336897 1066 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:50.336966 1066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:50.337109 1066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:50.338192 1066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "5c9d99471836494d960a3a321bf1e5b3"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "c775aad08dd1f9298187bcfc0eb9bfb9"
server_key_iv: "9f349980d951ae891f311fd0291e0414"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.338565 1066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "5c9d99471836494d960a3a321bf1e5b3"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "c775aad08dd1f9298187bcfc0eb9bfb9"
server_key_iv: "9f349980d951ae891f311fd0291e0414"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.339884 1066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:54:50.340655 1081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.340831 1066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:50.340937 1066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "5c9d99471836494d960a3a321bf1e5b3"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "c775aad08dd1f9298187bcfc0eb9bfb9"
server_key_iv: "9f349980d951ae891f311fd0291e0414"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.341015 1066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:50.352412 1066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:50.354406 1066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:50.354584 1066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:50.354833 1066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:50.355185 1066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:50.355250 1066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.355304 1066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:50.355355 1066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.361630 1066 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:34197
I20260619 08:54:50.361672 1194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:34197 every 8 connection(s)
I20260619 08:54:50.362010 1066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:54:50.368944 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 1066
I20260619 08:54:50.369081 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:54:50.369295 16192 external_mini_cluster.cc:1428] Setting key ed5f80faa7fbd303abad96d624939593
Jun 19 08:54:50 dist-test-slave-p0h9 krb5kdc[691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859290, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:50.373832 793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.362849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:36495 (local address 127.15.208.62:38817)
0619 08:54:50.362893 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.362895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.363111 (+ 216us) server_negotiation.cc:407] Connection header received
0619 08:54:50.363718 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.363719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.363734 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.363814 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:50.365418 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.366329 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.367930 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.368151 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.368232 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.370971 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:50.370972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:50.370972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:50.370979 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:50.372386 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:50.372822 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:50.372823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:50.372824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:50.372886 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:50.373163 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:50.373164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:50.373164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:50.373289 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:50.373368 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.373657 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.373740 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:54:50.374028 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.362823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:38817 (local address 127.15.208.3:36495)
0619 08:54:50.362996 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.363039 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.363608 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.363916 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.363921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.364913 (+ 992us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:50.365283 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.365291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.366439 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.366441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.367827 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.367830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.368239 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.368241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.368394 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.369085 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:50.369097 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:50.370890 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:50.372465 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:50.372471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:50.372472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:50.372706 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:50.372964 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:50.372965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:50.372966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:50.373088 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:50.373357 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:50.373363 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:50.373511 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.373623 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.373787 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":111,"threads_started":1}
I20260619 08:54:50.374565 1195 heartbeater.cc:347] Connected to a master server at 127.15.208.62:38817
I20260619 08:54:50.374670 1195 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:50.374812 1195 heartbeater.cc:511] Master 127.15.208.62:38817 requested a full tablet report, sending...
I20260619 08:54:50.375175 736 ts_manager.cc:194] Registered new tserver with Master: 5c9d99471836494d960a3a321bf1e5b3 (127.15.208.3:34197)
I20260619 08:54:50.375514 736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:36495
I20260619 08:54:50.381309 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:54:50.388970 793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.382679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50670 (local address 127.15.208.62:38817)
0619 08:54:50.382761 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.382764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.382858 (+ 94us) server_negotiation.cc:407] Connection header received
0619 08:54:50.382937 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.382938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.382949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.383000 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:50.383660 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.384166 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.384972 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.385189 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.385274 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.386497 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:50.386498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:50.386499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:50.386507 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:50.387690 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:50.388031 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:50.388033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:50.388033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:50.388106 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:50.388318 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:50.388319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:50.388319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:50.388479 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:50.388586 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.388777 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.388859 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 08:54:50.390894 736 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:54:50.392452 736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:54:50.399195 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.395345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45606 (local address 127.15.208.3:34197)
0619 08:54:50.395536 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.395540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.395549 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:54:50.395588 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.395590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.395604 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.395681 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:50.395792 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.396978 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.398265 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.398754 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.398841 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.398899 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.399061 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.399107 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260619 08:54:50.399216 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.394860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:34391 (local address 127.0.0.1:53492)
0619 08:54:50.395092 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.395139 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.395208 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.395701 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.395702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.395704 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:50.395741 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.395742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.396637 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.396638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.397922 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.397925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.398930 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.398931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.399010 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.399014 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.399062 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.399114 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260619 08:54:50.399233 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.394918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:44901 (local address 127.0.0.1:40846)
0619 08:54:50.395248 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.395259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.395310 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.395632 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.395635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.395638 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:50.395713 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.395715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.397124 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.397125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.398092 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.398095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.398990 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.398991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.399057 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.399060 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.399090 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.399120 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":24,"thread_start_us":111,"threads_started":1}
I20260619 08:54:50.399391 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.395066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.15.208.1:44901)
0619 08:54:50.395389 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.395393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.395405 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:54:50.395479 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.395482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.395500 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.395854 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:50.395925 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.397033 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.398185 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.398892 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.399165 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.399228 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.399255 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.399303 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":103,"threads_started":1}
I20260619 08:54:50.399484 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.395137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:34197 (local address 127.0.0.1:45606)
0619 08:54:50.395275 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.395284 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.395332 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.395679 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.395680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.395682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:50.395729 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.395730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.397116 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.397117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.398176 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.398178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.398840 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.398841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.398920 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.398933 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.399035 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.399093 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":26,"thread_start_us":60,"threads_started":1}
I20260619 08:54:50.399760 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.395082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.15.208.2:34391)
0619 08:54:50.395450 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.395455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.395465 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:50.395507 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.395509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.395525 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.395615 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:50.395815 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.396528 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.398010 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.398758 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.398831 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.398891 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.399409 (+ 518us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.399683 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":93,"threads_started":1}
I20260619 08:54:50.400116 861 tablet_service.cc:1467] Processing CreateTablet for tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3 (DEFAULT_TABLE table=test-table [id=b6589655dfc94446b395cd40d9583ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:50.400530 861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:50.400453 995 tablet_service.cc:1467] Processing CreateTablet for tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3 (DEFAULT_TABLE table=test-table [id=b6589655dfc94446b395cd40d9583ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:50.400766 995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:50.401362 1129 tablet_service.cc:1467] Processing CreateTablet for tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3 (DEFAULT_TABLE table=test-table [id=b6589655dfc94446b395cd40d9583ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:50.401695 1129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:50.403452 1212 tablet_bootstrap.cc:492] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: Bootstrap starting.
I20260619 08:54:50.404007 1213 tablet_bootstrap.cc:492] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972: Bootstrap starting.
I20260619 08:54:50.404160 1212 tablet_bootstrap.cc:654] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:50.404758 1213 tablet_bootstrap.cc:654] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:50.404776 1212 log.cc:824] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:50.404865 1214 tablet_bootstrap.cc:492] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: Bootstrap starting.
I20260619 08:54:50.405344 1213 log.cc:824] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:50.405545 1214 tablet_bootstrap.cc:654] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:50.405782 1212 tablet_bootstrap.cc:492] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: No bootstrap required, opened a new log
I20260619 08:54:50.405860 1212 ts_tablet_manager.cc:1397] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:54:50.406172 1213 tablet_bootstrap.cc:492] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972: No bootstrap required, opened a new log
I20260619 08:54:50.406198 1214 log.cc:824] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:50.406234 1213 ts_tablet_manager.cc:1397] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:54:50.406957 1214 tablet_bootstrap.cc:492] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: No bootstrap required, opened a new log
I20260619 08:54:50.407013 1214 ts_tablet_manager.cc:1397] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:54:50.407820 1213 raft_consensus.cc:348] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.407948 1213 raft_consensus.cc:374] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:50.407967 1213 raft_consensus.cc:729] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2145f8e91a0c47edaa3b18685dc9f972, State: Initialized, Role: FOLLOWER
I20260619 08:54:50.408041 1213 consensus_queue.cc:260] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.407964 1212 raft_consensus.cc:348] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.408126 1212 raft_consensus.cc:374] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:50.408164 1212 raft_consensus.cc:729] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58bba4d5ae334c45840b819bb355752b, State: Initialized, Role: FOLLOWER
I20260619 08:54:50.408293 927 heartbeater.cc:503] Master 127.15.208.62:38817 was elected leader, sending a full tablet report...
I20260619 08:54:50.408275 1212 consensus_queue.cc:260] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.408329 1213 ts_tablet_manager.cc:1428] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:54:50.408551 1212 ts_tablet_manager.cc:1428] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:54:50.408565 1061 heartbeater.cc:503] Master 127.15.208.62:38817 was elected leader, sending a full tablet report...
I20260619 08:54:50.409390 1214 raft_consensus.cc:348] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.409579 1214 raft_consensus.cc:374] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:50.409617 1214 raft_consensus.cc:729] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c9d99471836494d960a3a321bf1e5b3, State: Initialized, Role: FOLLOWER
I20260619 08:54:50.409718 1214 consensus_queue.cc:260] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.409960 1214 ts_tablet_manager.cc:1428] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:54:50.410007 1195 heartbeater.cc:503] Master 127.15.208.62:38817 was elected leader, sending a full tablet report...
W20260619 08:54:50.444262 1062 tablet.cc:2369] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:54:50.447209 1220 raft_consensus.cc:482] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:50.447368 1220 raft_consensus.cc:504] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.447633 1219 raft_consensus.cc:482] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:50.447721 1219 raft_consensus.cc:504] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.448077 1219 leader_election.cc:288] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2145f8e91a0c47edaa3b18685dc9f972 (127.15.208.1:44901), 5c9d99471836494d960a3a321bf1e5b3 (127.15.208.3:34197)
I20260619 08:54:50.448624 1220 leader_election.cc:288] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58bba4d5ae334c45840b819bb355752b (127.15.208.2:34391), 2145f8e91a0c47edaa3b18685dc9f972 (127.15.208.1:44901)
I20260619 08:54:50.451602 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.448102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:36803 (local address 127.15.208.1:44901)
0619 08:54:50.448195 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.448197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.448208 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:54:50.448244 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.448245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.448254 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.448336 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:50.448565 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.449325 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.450644 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.451227 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.451294 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.451303 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.451496 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.451534 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":70}
I20260619 08:54:50.451632 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.447987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:44901 (local address 127.15.208.3:36803)
0619 08:54:50.448064 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.448077 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.448122 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.448396 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.448397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.448399 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:50.448478 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.448480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.449420 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.449421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.450551 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.450554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.451336 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.451337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.451420 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.451423 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.451468 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.451561 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":46}
I20260619 08:54:50.451973 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.448151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:56931 (local address 127.15.208.2:34391)
0619 08:54:50.448201 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.448203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.448313 (+ 110us) server_negotiation.cc:407] Connection header received
0619 08:54:50.448413 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.448414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.448424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.448455 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:50.448692 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.449425 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.450792 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.451288 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.451328 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.451336 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.451869 (+ 533us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.451909 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":24}
I20260619 08:54:50.452100 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.448302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:44901 (local address 127.15.208.2:44615)
0619 08:54:50.448554 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.448564 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.448606 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.448812 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.448813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.448814 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:50.448859 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.448860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.449928 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.449929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.451084 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.451086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.451896 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.451897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.451963 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.451966 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.451997 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.452033 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238}
I20260619 08:54:50.452219 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.447987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:34391 (local address 127.15.208.3:56931)
0619 08:54:50.448257 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.448267 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.448337 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.448528 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.448528 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.448530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:50.448608 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.448609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.449522 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.449523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.450692 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.450695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.451397 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.451398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.451469 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.451786 (+ 317us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.452094 (+ 308us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.452140 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":83,"thread_start_us":116,"threads_started":1}
I20260619 08:54:50.452311 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.449067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:58497 (local address 127.15.208.3:34197)
0619 08:54:50.449114 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.449116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.449123 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:54:50.449142 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.449143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.449150 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.449195 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:50.449375 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.450070 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.451389 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.451941 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.451990 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.451997 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.452224 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.452257 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:54:50.452327 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.448415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:34197 (local address 127.15.208.2:58497)
0619 08:54:50.448849 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.448859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.448913 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.449231 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.449232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.449233 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:50.449299 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.449300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.450168 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.450169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.451300 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.451303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.452058 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.452059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.452139 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.452143 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.452213 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.452250 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":100,"threads_started":1}
I20260619 08:54:50.452299 881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c23fc5fb7f4c6e8d5b4063ef5894f3" candidate_uuid: "5c9d99471836494d960a3a321bf1e5b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2145f8e91a0c47edaa3b18685dc9f972" is_pre_election: true
I20260619 08:54:50.452453 881 raft_consensus.cc:2454] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c9d99471836494d960a3a321bf1e5b3 in term 0.
I20260619 08:54:50.452499 1015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c23fc5fb7f4c6e8d5b4063ef5894f3" candidate_uuid: "5c9d99471836494d960a3a321bf1e5b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58bba4d5ae334c45840b819bb355752b" is_pre_election: true
I20260619 08:54:50.452594 1015 raft_consensus.cc:2454] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c9d99471836494d960a3a321bf1e5b3 in term 0.
I20260619 08:54:50.452768 1082 leader_election.cc:302] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2145f8e91a0c47edaa3b18685dc9f972, 5c9d99471836494d960a3a321bf1e5b3; no voters:
I20260619 08:54:50.452811 1149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c23fc5fb7f4c6e8d5b4063ef5894f3" candidate_uuid: "58bba4d5ae334c45840b819bb355752b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c9d99471836494d960a3a321bf1e5b3" is_pre_election: true
I20260619 08:54:50.452889 1149 raft_consensus.cc:2454] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58bba4d5ae334c45840b819bb355752b in term 0.
I20260619 08:54:50.452924 1220 raft_consensus.cc:2781] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:54:50.452984 1220 raft_consensus.cc:482] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:54:50.453013 1220 raft_consensus.cc:3037] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:50.453087 948 leader_election.cc:302] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58bba4d5ae334c45840b819bb355752b, 5c9d99471836494d960a3a321bf1e5b3; no voters:
I20260619 08:54:50.453154 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.448359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:44615 (local address 127.15.208.1:44901)
0619 08:54:50.448532 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.448533 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.448663 (+ 130us) server_negotiation.cc:407] Connection header received
0619 08:54:50.448690 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.448690 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.448700 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.448918 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:50.448950 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.449830 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.451177 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.451808 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.453025 (+ 1217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.453035 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.453055 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.453090 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":114,"threads_started":1}
I20260619 08:54:50.453212 1219 raft_consensus.cc:2781] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:54:50.453256 1219 raft_consensus.cc:482] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:54:50.453275 1219 raft_consensus.cc:3037] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:50.453358 881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c23fc5fb7f4c6e8d5b4063ef5894f3" candidate_uuid: "58bba4d5ae334c45840b819bb355752b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2145f8e91a0c47edaa3b18685dc9f972" is_pre_election: true
I20260619 08:54:50.453457 881 raft_consensus.cc:2454] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58bba4d5ae334c45840b819bb355752b in term 0.
I20260619 08:54:50.454030 1219 raft_consensus.cc:504] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.454135 1219 leader_election.cc:288] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [CANDIDATE]: Term 1 election: Requested vote from peers 2145f8e91a0c47edaa3b18685dc9f972 (127.15.208.1:44901), 5c9d99471836494d960a3a321bf1e5b3 (127.15.208.3:34197)
I20260619 08:54:50.454284 881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c23fc5fb7f4c6e8d5b4063ef5894f3" candidate_uuid: "58bba4d5ae334c45840b819bb355752b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2145f8e91a0c47edaa3b18685dc9f972"
I20260619 08:54:50.454330 1149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c23fc5fb7f4c6e8d5b4063ef5894f3" candidate_uuid: "58bba4d5ae334c45840b819bb355752b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c9d99471836494d960a3a321bf1e5b3"
I20260619 08:54:50.454355 881 raft_consensus.cc:3037] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:50.454846 1220 raft_consensus.cc:504] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.454998 1220 leader_election.cc:288] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [CANDIDATE]: Term 1 election: Requested vote from peers 58bba4d5ae334c45840b819bb355752b (127.15.208.2:34391), 2145f8e91a0c47edaa3b18685dc9f972 (127.15.208.1:44901)
I20260619 08:54:50.455010 1149 raft_consensus.cc:2379] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 58bba4d5ae334c45840b819bb355752b in current term 1: Already voted for candidate 5c9d99471836494d960a3a321bf1e5b3 in this term.
I20260619 08:54:50.455255 1015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c23fc5fb7f4c6e8d5b4063ef5894f3" candidate_uuid: "5c9d99471836494d960a3a321bf1e5b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58bba4d5ae334c45840b819bb355752b"
I20260619 08:54:50.455303 881 raft_consensus.cc:2454] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58bba4d5ae334c45840b819bb355752b in term 1.
I20260619 08:54:50.455341 1015 raft_consensus.cc:2379] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c9d99471836494d960a3a321bf1e5b3 in current term 1: Already voted for candidate 58bba4d5ae334c45840b819bb355752b in this term.
I20260619 08:54:50.455413 880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c23fc5fb7f4c6e8d5b4063ef5894f3" candidate_uuid: "5c9d99471836494d960a3a321bf1e5b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2145f8e91a0c47edaa3b18685dc9f972"
I20260619 08:54:50.455440 948 leader_election.cc:302] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2145f8e91a0c47edaa3b18685dc9f972, 58bba4d5ae334c45840b819bb355752b; no voters:
I20260619 08:54:50.455488 880 raft_consensus.cc:2379] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c9d99471836494d960a3a321bf1e5b3 in current term 1: Already voted for candidate 58bba4d5ae334c45840b819bb355752b in this term.
I20260619 08:54:50.455521 1219 raft_consensus.cc:2781] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:50.455670 1082 leader_election.cc:302] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c9d99471836494d960a3a321bf1e5b3; no voters: 2145f8e91a0c47edaa3b18685dc9f972, 58bba4d5ae334c45840b819bb355752b
I20260619 08:54:50.455678 1219 raft_consensus.cc:686] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 1 LEADER]: Becoming Leader. State: Replica: 58bba4d5ae334c45840b819bb355752b, State: Running, Role: LEADER
I20260619 08:54:50.455777 1220 raft_consensus.cc:2726] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 08:54:50.455780 1219 consensus_queue.cc:237] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } }
I20260619 08:54:50.456599 735 catalog_manager.cc:5495] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b reported cstate change: term changed from 0 to 1, leader changed from <none> to 58bba4d5ae334c45840b819bb355752b (127.15.208.2). New cstate: current_term: 1 leader_uuid: "58bba4d5ae334c45840b819bb355752b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58bba4d5ae334c45840b819bb355752b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 34391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 } health_report { overall_health: UNKNOWN } } }
I20260619 08:54:50.474748 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.472208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.15.208.2:34391)
0619 08:54:50.472286 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.472288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.472294 (+ 6us) server_negotiation.cc:407] Connection header received
0619 08:54:50.472325 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.472327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.472343 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.472424 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:54:50.472595 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.473145 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.473917 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.474127 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.474205 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.474362 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:54:50.474469 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:54:50.474558 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.474598 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.474618 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 08:54:50.476567 1149 raft_consensus.cc:1261] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 1 FOLLOWER]: Refusing update from remote peer 58bba4d5ae334c45840b819bb355752b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:50.476586 880 raft_consensus.cc:1261] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 1 FOLLOWER]: Refusing update from remote peer 58bba4d5ae334c45840b819bb355752b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:50.476899 1224 consensus_queue.cc:1035] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2145f8e91a0c47edaa3b18685dc9f972" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:50.477059 1219 consensus_queue.cc:1035] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c9d99471836494d960a3a321bf1e5b3" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 34197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:50.481688 735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:50.481768 735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:50.482544 735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 564a9cf250bf4af1a4040aa9183da9bc: Sending DeleteTablet for 3 replicas of tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3
I20260619 08:54:50.482882 861 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:50 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:40846
I20260619 08:54:50.482923 1129 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:50 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:45606
I20260619 08:54:50.482929 995 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:50 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53492
I20260619 08:54:50.483142 1236 tablet_replica.cc:330] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972: stopping tablet replica
I20260619 08:54:50.483283 1238 tablet_replica.cc:330] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: stopping tablet replica
I20260619 08:54:50.483299 1236 raft_consensus.cc:2229] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:50.483371 1237 tablet_replica.cc:330] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: stopping tablet replica
I20260619 08:54:50.483402 1236 raft_consensus.cc:2258] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 2145f8e91a0c47edaa3b18685dc9f972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:50.483448 1238 raft_consensus.cc:2229] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:54:50.483471 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 798
I20260619 08:54:50.483512 1237 raft_consensus.cc:2229] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:50.483611 1237 raft_consensus.cc:2258] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:50.484227 1238 raft_consensus.cc:2258] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:50.484715 1238 ts_tablet_manager.cc:1905] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:50.485237 1237 ts_tablet_manager.cc:1905] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:50.486536 1238 ts_tablet_manager.cc:1918] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:50.486613 1238 log.cc:1196] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/25c23fc5fb7f4c6e8d5b4063ef5894f3
I20260619 08:54:50.486915 1238 ts_tablet_manager.cc:1939] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 58bba4d5ae334c45840b819bb355752b: Deleting consensus metadata
I20260619 08:54:50.486945 1237 ts_tablet_manager.cc:1918] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:50.487011 1237 log.cc:1196] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/25c23fc5fb7f4c6e8d5b4063ef5894f3
I20260619 08:54:50.487344 721 catalog_manager.cc:4841] TS 58bba4d5ae334c45840b819bb355752b (127.15.208.2:34391): tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3 (table test-table [id=b6589655dfc94446b395cd40d9583ad2]) successfully deleted
I20260619 08:54:50.487634 1237 ts_tablet_manager.cc:1939] T 25c23fc5fb7f4c6e8d5b4063ef5894f3 P 5c9d99471836494d960a3a321bf1e5b3: Deleting consensus metadata
I20260619 08:54:50.488006 720 catalog_manager.cc:4841] TS 5c9d99471836494d960a3a321bf1e5b3 (127.15.208.3:34197): tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3 (table test-table [id=b6589655dfc94446b395cd40d9583ad2]) successfully deleted
W20260619 08:54:50.488652 720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:44901 (error 108)
I20260619 08:54:50.489180 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.489057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:44901 (local address 127.0.0.1:40860)
0619 08:54:50.489097 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.489134 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260619 08:54:50.489290 720 catalog_manager.cc:4584] TS 2145f8e91a0c47edaa3b18685dc9f972 (127.15.208.1:44901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25c23fc5fb7f4c6e8d5b4063ef5894f3: Network error: Client connection negotiation failed: client connection to 127.15.208.1:44901: connect: Connection refused (error 111)
I20260619 08:54:50.489360 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 932
I20260619 08:54:50.494174 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 1066
I20260619 08:54:50.498374 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 704
2026-06-19T08:54:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1953 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T08:54:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:54:50Z Disabled control of system clock
I20260619 08:54:50.525789 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39949
--webserver_interface=127.15.208.62
--webserver_port=0
--builtin_ntp_servers=127.15.208.20:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:39949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:54:50.607455 1243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:50.607681 1243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:50.607704 1243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:50.608803 1243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:54:50.608840 1243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:50.608852 1243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:50.608865 1243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:54:50.608876 1243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:54:50.610158 1243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:39949
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:39949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:50.610348 1243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:50.610590 1243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:50.615350 1249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:50.615362 1248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:50.615624 1243 server_base.cc:1034] running on GCE node
W20260619 08:54:50.615697 1251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:50.615954 1243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:50.616346 1243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:50.617509 1243 hybrid_clock.cc:648] HybridClock initialized: now 1781859290617488 us; error 49 us; skew 500 ppm
I20260619 08:54:50.619042 1243 webserver.cc:460] Webserver started at http://127.15.208.62:38835/ using document root <none> and password file <none>
I20260619 08:54:50.619295 1243 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:50.619364 1243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:50.619513 1243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:50.620595 1243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "2d0df4bea8ad4d91813efe8de28dbd62"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "cb799b827add729b817669a572f0ef5a"
server_key_iv: "3e0419e218f62acbabfcce1b1ff5f853"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.621001 1243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "2d0df4bea8ad4d91813efe8de28dbd62"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "cb799b827add729b817669a572f0ef5a"
server_key_iv: "3e0419e218f62acbabfcce1b1ff5f853"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.622319 1243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:50.623051 1257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.623229 1243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:54:50.623327 1243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "2d0df4bea8ad4d91813efe8de28dbd62"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "cb799b827add729b817669a572f0ef5a"
server_key_iv: "3e0419e218f62acbabfcce1b1ff5f853"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.623409 1243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:50.631305 1243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:50.631577 1243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:50.631732 1243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:50.635591 1243 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:39949
I20260619 08:54:50.635627 1309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:39949 every 8 connection(s)
I20260619 08:54:50.635984 1243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:54:50.637028 1310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:50.639163 1310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62: Bootstrap starting.
I20260619 08:54:50.639714 1310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:50.639807 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 1243
I20260619 08:54:50.639945 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:54:50.640057 1310 log.cc:824] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:50.640148 16192 external_mini_cluster.cc:1428] Setting key e153b1a850f758b1ab5c438f58dac570
I20260619 08:54:50.641041 1310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62: No bootstrap required, opened a new log
I20260619 08:54:50.642752 1310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df4bea8ad4d91813efe8de28dbd62" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39949 } }
I20260619 08:54:50.642894 1310 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:50.642916 1310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d0df4bea8ad4d91813efe8de28dbd62, State: Initialized, Role: FOLLOWER
I20260619 08:54:50.643078 1310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df4bea8ad4d91813efe8de28dbd62" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39949 } }
I20260619 08:54:50.643181 1310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:54:50.643260 1310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:54:50.643312 1310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:50.644054 1310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df4bea8ad4d91813efe8de28dbd62" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39949 } }
I20260619 08:54:50.644171 1310 leader_election.cc:302] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d0df4bea8ad4d91813efe8de28dbd62; no voters:
I20260619 08:54:50.644349 1310 leader_election.cc:288] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:54:50.644531 1316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:50.644614 1310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:54:50.644706 1316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [term 1 LEADER]: Becoming Leader. State: Replica: 2d0df4bea8ad4d91813efe8de28dbd62, State: Running, Role: LEADER
I20260619 08:54:50.644881 1316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df4bea8ad4d91813efe8de28dbd62" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39949 } }
I20260619 08:54:50.645220 1316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d0df4bea8ad4d91813efe8de28dbd62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df4bea8ad4d91813efe8de28dbd62" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39949 } } }
I20260619 08:54:50.645303 1316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:50.645419 1316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d0df4bea8ad4d91813efe8de28dbd62. Latest consensus state: current_term: 1 leader_uuid: "2d0df4bea8ad4d91813efe8de28dbd62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df4bea8ad4d91813efe8de28dbd62" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 39949 } } }
I20260619 08:54:50.645471 1316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62 [sys.catalog]: This master's current role is: LEADER
W20260619 08:54:50.647038 1330 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:54:50.647092 1330 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:54:50.647142 1321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:54:50.647311 1321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:54:50.648859 1321 catalog_manager.cc:1347] Generated new cluster ID: f24e694597f54a058a672e20ed9ceecd
I20260619 08:54:50.648909 1321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:54:50.649348 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.640750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42056 (local address 127.15.208.62:39949)
0619 08:54:50.643177 (+ 2427us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.643186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.643221 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:54:50.643351 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.643358 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.643435 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.643588 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:50.643847 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.645761 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.646968 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.647595 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.647718 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.648507 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:50.648511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:50.648512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:50.648718 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:50.648787 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:50.648802 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:50.648870 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.649067 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.649145 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2331,"thread_start_us":76,"threads_started":1}
I20260619 08:54:50.659926 1321 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:54:50.660472 1321 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:54:50.665750 1321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62: Generated new TSK 0
I20260619 08:54:50.665994 1321 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:54:50.669351 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39949
--builtin_ntp_servers=127.15.208.20:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:54:50.752589 1334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:50.752861 1334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:50.752902 1334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:50.754061 1334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:50.754127 1334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:50.754166 1334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:54:50.755501 1334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:50.755771 1334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:50.756085 1334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:50.761920 1340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:50.761936 1342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:50.762048 1339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:50.762287 1334 server_base.cc:1034] running on GCE node
I20260619 08:54:50.762508 1334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:50.762771 1334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:50.763928 1334 hybrid_clock.cc:648] HybridClock initialized: now 1781859290763918 us; error 29 us; skew 500 ppm
I20260619 08:54:50.765304 1334 webserver.cc:460] Webserver started at http://127.15.208.1:38291/ using document root <none> and password file <none>
I20260619 08:54:50.765573 1334 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:50.765620 1334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:50.765761 1334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:50.766817 1334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "d12dc2d24c594a6f9a43716562583fed"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "1ce755c510df646b8b87ed8dbcb653b3"
server_key_iv: "50cea695d472c93122d964c7ee43b2f2"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.767177 1334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "d12dc2d24c594a6f9a43716562583fed"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "1ce755c510df646b8b87ed8dbcb653b3"
server_key_iv: "50cea695d472c93122d964c7ee43b2f2"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.768487 1334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:54:50.769356 1348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.769536 1334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:50.769642 1334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "d12dc2d24c594a6f9a43716562583fed"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "1ce755c510df646b8b87ed8dbcb653b3"
server_key_iv: "50cea695d472c93122d964c7ee43b2f2"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.769716 1334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:50.816885 1334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:50.817193 1334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:50.817297 1334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:50.817494 1334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:50.817821 1334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:50.817854 1334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.817906 1334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:50.817934 1334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.823908 1334 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:41443
I20260619 08:54:50.823951 1461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:41443 every 8 connection(s)
I20260619 08:54:50.824285 1334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:54:50.826259 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 1334
I20260619 08:54:50.826383 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:54:50.826606 16192 external_mini_cluster.cc:1428] Setting key 36cd7fef3af54e41a1adc7a7969c7999
I20260619 08:54:50.827621 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39949
--builtin_ntp_servers=127.15.208.20:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:54:50.830367 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.825098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:49095 (local address 127.15.208.62:39949)
0619 08:54:50.825154 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.825158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.825403 (+ 245us) server_negotiation.cc:407] Connection header received
0619 08:54:50.825964 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.825966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.825993 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.826049 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:50.827025 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.827625 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.828832 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.829040 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.829132 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.830075 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:50.830076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:50.830077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:50.830081 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:50.830098 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:50.830104 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:50.830176 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.830255 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.830288 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:54:50.830642 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.825074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39949 (local address 127.15.208.1:49095)
0619 08:54:50.825276 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.825310 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.825835 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.826131 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.826135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.826151 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:50.826926 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.826934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.827835 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.827836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.828684 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.828686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.829142 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.829143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.829288 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.829900 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:54:50.829901 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:50.829930 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:54:50.829953 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:50.830178 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:50.830185 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:50.830188 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.830259 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.830361 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260619 08:54:50.831108 1462 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39949
I20260619 08:54:50.831197 1462 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:50.831391 1462 heartbeater.cc:511] Master 127.15.208.62:39949 requested a full tablet report, sending...
I20260619 08:54:50.831907 1274 ts_manager.cc:194] Registered new tserver with Master: d12dc2d24c594a6f9a43716562583fed (127.15.208.1:41443)
I20260619 08:54:50.832540 1274 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.1:49095
W20260619 08:54:50.910033 1465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:50.910274 1465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:50.910313 1465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:50.911438 1465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:50.911499 1465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:50.911541 1465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:54:50.912976 1465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:50.913239 1465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:50.913535 1465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:50.918346 1471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:50.918354 1470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:50.918354 1473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:50.918732 1465 server_base.cc:1034] running on GCE node
I20260619 08:54:50.918937 1465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:50.919200 1465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:50.920363 1465 hybrid_clock.cc:648] HybridClock initialized: now 1781859290920354 us; error 34 us; skew 500 ppm
I20260619 08:54:50.921643 1465 webserver.cc:460] Webserver started at http://127.15.208.2:44839/ using document root <none> and password file <none>
I20260619 08:54:50.921886 1465 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:50.921953 1465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:50.922091 1465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:50.923170 1465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "c602837b229b47c6a3799c94a2055264"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "837537a8eb745253b0c232fbfd23cbb2"
server_key_iv: "31043fa1578988116696502e6bda6c5b"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.923583 1465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "c602837b229b47c6a3799c94a2055264"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "837537a8eb745253b0c232fbfd23cbb2"
server_key_iv: "31043fa1578988116696502e6bda6c5b"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.924882 1465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:54:50.925678 1479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.925846 1465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:50.925962 1465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "c602837b229b47c6a3799c94a2055264"
format_stamp: "Formatted at 2026-06-19 08:54:50 on dist-test-slave-p0h9"
server_key: "837537a8eb745253b0c232fbfd23cbb2"
server_key_iv: "31043fa1578988116696502e6bda6c5b"
server_key_version: "encryptionkey@0"
I20260619 08:54:50.926050 1465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:50.942960 1465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:50.943311 1465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:50.943465 1465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:50.943712 1465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:50.944033 1465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:50.944088 1465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.944130 1465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:50.944154 1465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:50.950150 1465 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:37705
I20260619 08:54:50.950192 1592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:37705 every 8 connection(s)
I20260619 08:54:50.950560 1465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:54:50.952883 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 1465
I20260619 08:54:50.952975 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:54:50.953166 16192 external_mini_cluster.cc:1428] Setting key a95f1d82c15e78799ae818d1d709e198
I20260619 08:54:50.954159 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39949
--builtin_ntp_servers=127.15.208.20:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:54:50.956380 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.951301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:50961 (local address 127.15.208.62:39949)
0619 08:54:50.951366 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:50.951368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:50.951627 (+ 259us) server_negotiation.cc:407] Connection header received
0619 08:54:50.952154 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:50.952155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:50.952167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:50.952226 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:50.953048 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.953805 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.954929 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.955145 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.955234 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.956081 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:50.956082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:50.956083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:50.956088 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:50.956103 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:50.956108 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:50.956162 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:50.956276 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 08:54:50.956305 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:54:50.956609 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:50.951288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39949 (local address 127.15.208.2:50961)
0619 08:54:50.951501 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:50.951535 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:50.952042 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:50.952306 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:50.952309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:50.952318 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:50.952911 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.952916 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.953929 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.953930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.954837 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:50.954839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:50.955244 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:50.955245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:50.955378 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:50.955944 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:54:50.955944 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:50.955970 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:54:50.955991 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:50.956165 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:50.956170 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:50.956173 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:50.956278 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 08:54:50.956372 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260619 08:54:50.957109 1593 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39949
I20260619 08:54:50.957213 1593 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:50.957448 1593 heartbeater.cc:511] Master 127.15.208.62:39949 requested a full tablet report, sending...
I20260619 08:54:50.957816 1274 ts_manager.cc:194] Registered new tserver with Master: c602837b229b47c6a3799c94a2055264 (127.15.208.2:37705)
I20260619 08:54:50.958206 1274 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.2:50961
W20260619 08:54:51.035737 1596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:51.035946 1596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:51.035969 1596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:51.037179 1596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:51.037225 1596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:51.037252 1596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:54:51.038589 1596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:39949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:51.038846 1596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:51.039165 1596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:51.044111 1601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:51.044092 1596 server_base.cc:1034] running on GCE node
W20260619 08:54:51.044092 1602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:51.044078 1604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:51.044620 1596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:51.044922 1596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:51.046051 1596 hybrid_clock.cc:648] HybridClock initialized: now 1781859291046038 us; error 31 us; skew 500 ppm
I20260619 08:54:51.047333 1596 webserver.cc:460] Webserver started at http://127.15.208.3:36993/ using document root <none> and password file <none>
I20260619 08:54:51.047622 1596 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:51.047677 1596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:51.047869 1596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:51.049144 1596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "ea34048e6bb0498b91cf32ac34684ed4"
format_stamp: "Formatted at 2026-06-19 08:54:51 on dist-test-slave-p0h9"
server_key: "3d73312909be4354fd6d0555897aecaa"
server_key_iv: "6970f9a3acebdfec6de42c934c507689"
server_key_version: "encryptionkey@0"
I20260619 08:54:51.049546 1596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "ea34048e6bb0498b91cf32ac34684ed4"
format_stamp: "Formatted at 2026-06-19 08:54:51 on dist-test-slave-p0h9"
server_key: "3d73312909be4354fd6d0555897aecaa"
server_key_iv: "6970f9a3acebdfec6de42c934c507689"
server_key_version: "encryptionkey@0"
I20260619 08:54:51.051069 1596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:54:51.051849 1610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:51.052032 1596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:54:51.052124 1596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "ea34048e6bb0498b91cf32ac34684ed4"
format_stamp: "Formatted at 2026-06-19 08:54:51 on dist-test-slave-p0h9"
server_key: "3d73312909be4354fd6d0555897aecaa"
server_key_iv: "6970f9a3acebdfec6de42c934c507689"
server_key_version: "encryptionkey@0"
I20260619 08:54:51.052204 1596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:51.075109 1596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:51.075459 1596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:51.075620 1596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:51.075870 1596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:51.076260 1596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:51.076316 1596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:51.076359 1596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:51.076386 1596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:51.082130 1596 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:43875
I20260619 08:54:51.082206 1723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:43875 every 8 connection(s)
I20260619 08:54:51.082557 1596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:54:51.087630 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.083339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:48667 (local address 127.15.208.62:39949)
0619 08:54:51.083439 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:51.083441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:51.083672 (+ 231us) server_negotiation.cc:407] Connection header received
0619 08:54:51.084146 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:51.084147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:51.084158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:51.084222 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:51.084947 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.085431 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.086369 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.086544 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.086596 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.087415 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:51.087416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:51.087416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:51.087419 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:51.087429 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:51.087432 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:51.087469 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:51.087552 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 08:54:51.087572 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 08:54:51.087795 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.083314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:39949 (local address 127.15.208.3:48667)
0619 08:54:51.083526 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:51.083570 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:51.084062 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:51.084277 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:51.084280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:51.084288 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:51.084870 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.084876 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.085509 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.085510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.086297 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.086298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.086617 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.086618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.086722 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.087292 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:54:51.087292 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:51.087321 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:54:51.087343 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:51.087480 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:51.087483 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:51.087485 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:54:51.087531 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 08:54:51.087607 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260619 08:54:51.088333 1724 heartbeater.cc:347] Connected to a master server at 127.15.208.62:39949
I20260619 08:54:51.088433 1724 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:51.088615 1724 heartbeater.cc:511] Master 127.15.208.62:39949 requested a full tablet report, sending...
I20260619 08:54:51.089030 1274 ts_manager.cc:194] Registered new tserver with Master: ea34048e6bb0498b91cf32ac34684ed4 (127.15.208.3:43875)
I20260619 08:54:51.089388 1274 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.3:48667
I20260619 08:54:51.089623 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 1596
I20260619 08:54:51.089735 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:54:51.089926 16192 external_mini_cluster.cc:1428] Setting key 17591b032394697ed7472f7fa350c680
I20260619 08:54:51.090806 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:54:51.094781 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.092117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.15.208.62:39949)
0619 08:54:51.092158 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:51.092160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:51.092317 (+ 157us) server_negotiation.cc:407] Connection header received
0619 08:54:51.092338 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:51.092339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:51.092349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:51.092394 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:51.092507 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.093026 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.093708 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.093875 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.093922 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.094582 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:51.094582 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:51.094583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:51.094586 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:51.094596 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:51.094599 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:51.094636 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:51.094670 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 08:54:51.094700 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:54:51.096267 1274 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:54:51.097851 1274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:54:51.104300 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.100120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:41443 (local address 127.0.0.1:60758)
0619 08:54:51.100421 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:51.100432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:51.100493 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:51.100916 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:51.100920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:51.100922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:51.100996 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.100998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.101970 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.101971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.103211 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.103214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.104015 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.104016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.104091 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.104104 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 08:54:51.104175 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:54:51.104226 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":99,"threads_started":1}
I20260619 08:54:51.104317 1734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.100230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60758 (local address 127.15.208.1:41443)
0619 08:54:51.100699 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:51.100705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:51.100718 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:54:51.100790 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:51.100793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:51.100810 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:51.100896 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:51.101067 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.101887 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.103312 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.103923 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.104014 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.104116 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:51.104187 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 08:54:51.104226 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":95,"threads_started":1}
I20260619 08:54:51.104586 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.100068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:43875 (local address 127.0.0.1:41050)
0619 08:54:51.100275 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:51.100332 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:51.100408 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:51.100946 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:51.100947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:51.100949 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:51.101011 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.101013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.102316 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.102317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.103413 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.103415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.104341 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.104342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.104401 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.104405 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:54:51.104436 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 08:54:51.104474 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":77,"thread_start_us":92,"threads_started":1}
I20260619 08:54:51.104797 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.100426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38642 (local address 127.15.208.2:37705)
0619 08:54:51.100639 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:51.100644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:51.101127 (+ 483us) server_negotiation.cc:407] Connection header received
0619 08:54:51.101171 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:51.101173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:51.101190 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:51.101266 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:51.101388 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.102411 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.103577 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.104269 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.104332 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.104416 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:51.104625 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 08:54:51.104674 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":106,"threads_started":1}
I20260619 08:54:51.104952 1735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.100321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:37705 (local address 127.0.0.1:38642)
0619 08:54:51.100997 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:51.101007 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:51.101057 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:51.101280 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:51.101281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:51.101284 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:51.101325 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.101326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.102491 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.102492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.103502 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.103505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.104447 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.104448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.104495 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.104498 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:51.104544 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 08:54:51.104575 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":85,"threads_started":1}
I20260619 08:54:51.105131 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.100469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.15.208.3:43875)
0619 08:54:51.100779 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:51.100783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:51.100792 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:54:51.100825 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:51.100827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:51.100843 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:51.101090 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:51.101149 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.102229 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.103506 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.104231 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.104865 (+ 634us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.104962 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:51.104988 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 08:54:51.105034 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":118,"threads_started":1}
I20260619 08:54:51.105173 1396 tablet_service.cc:1467] Processing CreateTablet for tablet 07569e3c466a421d84d9bf4dc7505bf9 (DEFAULT_TABLE table=test-table [id=f45bde1e2c28464685b6c0c98bc86ccf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:51.105566 1396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07569e3c466a421d84d9bf4dc7505bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:51.105569 1527 tablet_service.cc:1467] Processing CreateTablet for tablet 07569e3c466a421d84d9bf4dc7505bf9 (DEFAULT_TABLE table=test-table [id=f45bde1e2c28464685b6c0c98bc86ccf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:51.105818 1527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07569e3c466a421d84d9bf4dc7505bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:51.105741 1658 tablet_service.cc:1467] Processing CreateTablet for tablet 07569e3c466a421d84d9bf4dc7505bf9 (DEFAULT_TABLE table=test-table [id=f45bde1e2c28464685b6c0c98bc86ccf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:51.106041 1658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07569e3c466a421d84d9bf4dc7505bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:51.108243 1738 tablet_bootstrap.cc:492] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: Bootstrap starting.
I20260619 08:54:51.108906 1738 tablet_bootstrap.cc:654] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:51.109299 1738 log.cc:824] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:51.109428 1739 tablet_bootstrap.cc:492] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: Bootstrap starting.
I20260619 08:54:51.109730 1740 tablet_bootstrap.cc:492] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed: Bootstrap starting.
I20260619 08:54:51.110062 1739 tablet_bootstrap.cc:654] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:51.110242 1740 tablet_bootstrap.cc:654] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:51.110396 1739 log.cc:824] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:51.110466 1740 log.cc:824] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:51.110970 1738 tablet_bootstrap.cc:492] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: No bootstrap required, opened a new log
I20260619 08:54:51.111023 1738 ts_tablet_manager.cc:1397] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 08:54:51.111117 1739 tablet_bootstrap.cc:492] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: No bootstrap required, opened a new log
I20260619 08:54:51.111203 1739 ts_tablet_manager.cc:1397] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:54:51.113117 1740 tablet_bootstrap.cc:492] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed: No bootstrap required, opened a new log
I20260619 08:54:51.113052 1738 raft_consensus.cc:348] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea34048e6bb0498b91cf32ac34684ed4" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 43875 } } peers { permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 } } peers { permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 } }
I20260619 08:54:51.113183 1740 ts_tablet_manager.cc:1397] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 08:54:51.113266 1738 raft_consensus.cc:374] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:51.113322 1738 raft_consensus.cc:729] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c602837b229b47c6a3799c94a2055264, State: Initialized, Role: FOLLOWER
I20260619 08:54:51.113297 1739 raft_consensus.cc:348] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea34048e6bb0498b91cf32ac34684ed4" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 43875 } } peers { permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 } } peers { permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 } }
I20260619 08:54:51.113457 1739 raft_consensus.cc:374] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:51.113442 1738 consensus_queue.cc:260] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea34048e6bb0498b91cf32ac34684ed4" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 43875 } } peers { permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 } } peers { permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 } }
I20260619 08:54:51.113533 1739 raft_consensus.cc:729] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea34048e6bb0498b91cf32ac34684ed4, State: Initialized, Role: FOLLOWER
I20260619 08:54:51.113662 1739 consensus_queue.cc:260] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea34048e6bb0498b91cf32ac34684ed4" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 43875 } } peers { permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 } } peers { permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 } }
I20260619 08:54:51.113756 1738 ts_tablet_manager.cc:1428] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:54:51.113795 1593 heartbeater.cc:503] Master 127.15.208.62:39949 was elected leader, sending a full tablet report...
I20260619 08:54:51.113976 1739 ts_tablet_manager.cc:1428] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:54:51.114126 1724 heartbeater.cc:503] Master 127.15.208.62:39949 was elected leader, sending a full tablet report...
I20260619 08:54:51.115857 1740 raft_consensus.cc:348] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea34048e6bb0498b91cf32ac34684ed4" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 43875 } } peers { permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 } } peers { permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 } }
I20260619 08:54:51.115974 1740 raft_consensus.cc:374] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:51.115995 1740 raft_consensus.cc:729] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12dc2d24c594a6f9a43716562583fed, State: Initialized, Role: FOLLOWER
I20260619 08:54:51.116137 1740 consensus_queue.cc:260] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea34048e6bb0498b91cf32ac34684ed4" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 43875 } } peers { permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 } } peers { permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 } }
I20260619 08:54:51.116394 1740 ts_tablet_manager.cc:1428] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 08:54:51.116470 1462 heartbeater.cc:503] Master 127.15.208.62:39949 was elected leader, sending a full tablet report...
W20260619 08:54:51.201107 1594 tablet.cc:2369] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:54:51.209810 1745 raft_consensus.cc:482] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:51.209939 1745 raft_consensus.cc:504] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea34048e6bb0498b91cf32ac34684ed4" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 43875 } } peers { permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 } } peers { permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 } }
I20260619 08:54:51.210397 1745 leader_election.cc:288] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d12dc2d24c594a6f9a43716562583fed (127.15.208.1:41443), c602837b229b47c6a3799c94a2055264 (127.15.208.2:37705)
I20260619 08:54:51.213419 1734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.210651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:46545 (local address 127.15.208.1:41443)
0619 08:54:51.210700 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:51.210702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:51.210714 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:54:51.210810 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:51.210812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:51.210833 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:51.210902 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:51.211069 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.211587 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.212643 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.213036 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.213127 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.213135 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:51.213299 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 08:54:51.213344 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:54:51.213419 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.210738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:33405 (local address 127.15.208.2:37705)
0619 08:54:51.210787 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:51.210789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:51.210878 (+ 89us) server_negotiation.cc:407] Connection header received
0619 08:54:51.210982 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:51.210983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:51.210995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:51.211069 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:51.211225 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.211687 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.212650 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.213065 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.213149 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.213156 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:51.213312 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 08:54:51.213350 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:54:51.213440 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.210578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:41443 (local address 127.15.208.3:46545)
0619 08:54:51.210636 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:51.210648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:51.210701 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:51.210908 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:51.210908 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:51.210910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:51.210983 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.210985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.211679 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.211680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.212541 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.212544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.213125 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.213126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.213211 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.213214 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:51.213290 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:54:51.213361 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":25}
I20260619 08:54:51.213452 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.210599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:37705 (local address 127.15.208.3:33405)
0619 08:54:51.210788 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:51.210799 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:51.210893 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:51.211075 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:51.211075 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:51.211077 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:51.211155 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.211156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.211776 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.211777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.212556 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:51.212558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.213150 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.213151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:51.213228 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.213231 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:51.213293 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 08:54:51.213351 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":4,"thread_start_us":101,"threads_started":1}
I20260619 08:54:51.213894 1416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07569e3c466a421d84d9bf4dc7505bf9" candidate_uuid: "ea34048e6bb0498b91cf32ac34684ed4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d12dc2d24c594a6f9a43716562583fed" is_pre_election: true
I20260619 08:54:51.214087 1416 raft_consensus.cc:2454] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea34048e6bb0498b91cf32ac34684ed4 in term 0.
I20260619 08:54:51.214156 1547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07569e3c466a421d84d9bf4dc7505bf9" candidate_uuid: "ea34048e6bb0498b91cf32ac34684ed4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c602837b229b47c6a3799c94a2055264" is_pre_election: true
I20260619 08:54:51.214306 1547 raft_consensus.cc:2454] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea34048e6bb0498b91cf32ac34684ed4 in term 0.
I20260619 08:54:51.214372 1611 leader_election.cc:302] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d12dc2d24c594a6f9a43716562583fed, ea34048e6bb0498b91cf32ac34684ed4; no voters:
I20260619 08:54:51.214519 1745 raft_consensus.cc:2781] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:54:51.214574 1745 raft_consensus.cc:482] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:54:51.214598 1745 raft_consensus.cc:3037] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:51.215302 1745 raft_consensus.cc:504] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea34048e6bb0498b91cf32ac34684ed4" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 43875 } } peers { permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 } } peers { permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 } }
I20260619 08:54:51.215508 1745 leader_election.cc:288] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [CANDIDATE]: Term 1 election: Requested vote from peers d12dc2d24c594a6f9a43716562583fed (127.15.208.1:41443), c602837b229b47c6a3799c94a2055264 (127.15.208.2:37705)
I20260619 08:54:51.215590 1547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07569e3c466a421d84d9bf4dc7505bf9" candidate_uuid: "ea34048e6bb0498b91cf32ac34684ed4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c602837b229b47c6a3799c94a2055264"
I20260619 08:54:51.215588 1416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07569e3c466a421d84d9bf4dc7505bf9" candidate_uuid: "ea34048e6bb0498b91cf32ac34684ed4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d12dc2d24c594a6f9a43716562583fed"
I20260619 08:54:51.215667 1547 raft_consensus.cc:3037] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:51.215667 1416 raft_consensus.cc:3037] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:51.216599 1547 raft_consensus.cc:2454] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea34048e6bb0498b91cf32ac34684ed4 in term 1.
I20260619 08:54:51.216594 1416 raft_consensus.cc:2454] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea34048e6bb0498b91cf32ac34684ed4 in term 1.
I20260619 08:54:51.216827 1613 leader_election.cc:302] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c602837b229b47c6a3799c94a2055264, ea34048e6bb0498b91cf32ac34684ed4; no voters:
I20260619 08:54:51.216930 1745 raft_consensus.cc:2781] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:51.217070 1745 raft_consensus.cc:686] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 1 LEADER]: Becoming Leader. State: Replica: ea34048e6bb0498b91cf32ac34684ed4, State: Running, Role: LEADER
I20260619 08:54:51.217175 1745 consensus_queue.cc:237] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea34048e6bb0498b91cf32ac34684ed4" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 43875 } } peers { permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 } } peers { permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 } }
I20260619 08:54:51.218010 1274 catalog_manager.cc:5495] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea34048e6bb0498b91cf32ac34684ed4 (127.15.208.3). New cstate: current_term: 1 leader_uuid: "ea34048e6bb0498b91cf32ac34684ed4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea34048e6bb0498b91cf32ac34684ed4" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 43875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 } health_report { overall_health: UNKNOWN } } }
I20260619 08:54:51.223176 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.220817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.15.208.3:43875)
0619 08:54:51.220858 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:51.220861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:51.220870 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:54:51.220924 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:51.220925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:51.220941 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:51.221022 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:54:51.221164 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.221612 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.222388 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:51.222598 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:51.222674 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:51.222835 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:54:51.222936 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:54:51.223028 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:51.223048 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 08:54:51.223068 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:54:51.224982 1416 raft_consensus.cc:1261] T 07569e3c466a421d84d9bf4dc7505bf9 P d12dc2d24c594a6f9a43716562583fed [term 1 FOLLOWER]: Refusing update from remote peer ea34048e6bb0498b91cf32ac34684ed4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:51.224992 1547 raft_consensus.cc:1261] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264 [term 1 FOLLOWER]: Refusing update from remote peer ea34048e6bb0498b91cf32ac34684ed4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:51.225370 1748 consensus_queue.cc:1035] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c602837b229b47c6a3799c94a2055264" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 37705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:51.225601 1745 consensus_queue.cc:1035] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d12dc2d24c594a6f9a43716562583fed" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 41443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:51.228233 1753 mvcc.cc:205] Tried to move back new op lower bound from 7298495656854532096 to 7298495656826302464. Current Snapshot: MvccSnapshot[applied={T|T < 7298495656854532096}]
I20260619 08:54:51.228405 1752 mvcc.cc:205] Tried to move back new op lower bound from 7298495656854532096 to 7298495656826302464. Current Snapshot: MvccSnapshot[applied={T|T < 7298495656854532096}]
I20260619 08:54:51.230669 1274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:51.230767 1274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:51.231547 1274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d0df4bea8ad4d91813efe8de28dbd62: Sending DeleteTablet for 3 replicas of tablet 07569e3c466a421d84d9bf4dc7505bf9
I20260619 08:54:51.231840 1658 tablet_service.cc:1514] Processing DeleteTablet for tablet 07569e3c466a421d84d9bf4dc7505bf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:51 UTC) from {username='slave'} at 127.0.0.1:41050
I20260619 08:54:51.231904 1396 tablet_service.cc:1514] Processing DeleteTablet for tablet 07569e3c466a421d84d9bf4dc7505bf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:51 UTC) from {username='slave'} at 127.0.0.1:60758
I20260619 08:54:51.231899 1527 tablet_service.cc:1514] Processing DeleteTablet for tablet 07569e3c466a421d84d9bf4dc7505bf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:51 UTC) from {username='slave'} at 127.0.0.1:38642
I20260619 08:54:51.232227 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 1334
I20260619 08:54:51.234555 1760 tablet_replica.cc:330] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: stopping tablet replica
I20260619 08:54:51.234640 1760 raft_consensus.cc:2229] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:54:51.234791 1760 raft_consensus.cc:2258] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:51.234978 1761 tablet_replica.cc:330] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: stopping tablet replica
I20260619 08:54:51.235097 1761 raft_consensus.cc:2229] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:51.235210 1761 raft_consensus.cc:2258] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:51.235301 1760 ts_tablet_manager.cc:1905] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:51.235636 1761 ts_tablet_manager.cc:1905] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:51.237138 1760 ts_tablet_manager.cc:1918] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:51.237221 1760 log.cc:1196] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/07569e3c466a421d84d9bf4dc7505bf9
I20260619 08:54:51.237249 1761 ts_tablet_manager.cc:1918] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:51.237346 1761 log.cc:1196] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/07569e3c466a421d84d9bf4dc7505bf9
I20260619 08:54:51.237526 1760 ts_tablet_manager.cc:1939] T 07569e3c466a421d84d9bf4dc7505bf9 P ea34048e6bb0498b91cf32ac34684ed4: Deleting consensus metadata
I20260619 08:54:51.237582 1761 ts_tablet_manager.cc:1939] T 07569e3c466a421d84d9bf4dc7505bf9 P c602837b229b47c6a3799c94a2055264: Deleting consensus metadata
I20260619 08:54:51.237957 1260 catalog_manager.cc:4841] TS c602837b229b47c6a3799c94a2055264 (127.15.208.2:37705): tablet 07569e3c466a421d84d9bf4dc7505bf9 (table test-table [id=f45bde1e2c28464685b6c0c98bc86ccf]) successfully deleted
I20260619 08:54:51.238062 1260 catalog_manager.cc:4841] TS ea34048e6bb0498b91cf32ac34684ed4 (127.15.208.3:43875): tablet 07569e3c466a421d84d9bf4dc7505bf9 (table test-table [id=f45bde1e2c28464685b6c0c98bc86ccf]) successfully deleted
W20260619 08:54:51.243752 1258 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:41443 (error 108)
I20260619 08:54:51.244308 1735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:51.244142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:41443 (local address 127.0.0.1:60760)
0619 08:54:51.244204 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:51.244249 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 08:54:51.244382 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 1465
W20260619 08:54:51.245911 1258 catalog_manager.cc:4584] TS d12dc2d24c594a6f9a43716562583fed (127.15.208.1:41443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07569e3c466a421d84d9bf4dc7505bf9: Network error: Client connection negotiation failed: client connection to 127.15.208.1:41443: connect: Connection refused (error 111)
I20260619 08:54:51.249392 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 1596
I20260619 08:54:51.253979 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 1243
2026-06-19T08:54:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (755 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:54:51 dist-test-slave-p0h9 krb5kdc[1766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:54:51 dist-test-slave-p0h9 krb5kdc[1766](info): set up 2 sockets
Jun 19 08:54:51 dist-test-slave-p0h9 krb5kdc[1766](info): commencing operation
krb5kdc: starting...
W20260619 08:54:52.285113 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:54:52 dist-test-slave-p0h9 krb5kdc[1766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:54:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:54:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:54:52.405814 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:45951
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:52.485913 1779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:52.486121 1779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:52.486143 1779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:52.487210 1779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:54:52.487246 1779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:52.487259 1779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:52.487272 1779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:54:52.487283 1779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:54:52.488667 1779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:45951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:52.488878 1779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:52.489069 1779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:52.493873 1784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:52.493867 1787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:52.493973 1779 server_base.cc:1034] running on GCE node
W20260619 08:54:52.493867 1785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:52.494345 1779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:52.494683 1779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:52.495847 1779 hybrid_clock.cc:648] HybridClock initialized: now 1781859292495843 us; error 37 us; skew 500 ppm
Jun 19 08:54:52 dist-test-slave-p0h9 krb5kdc[1766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859292, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:52.498358 1779 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:54:52.498795 1779 webserver.cc:460] Webserver started at http://127.15.208.62:36065/ using document root <none> and password file <none>
I20260619 08:54:52.499027 1779 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:52.499096 1779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:52.499246 1779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:52.500350 1779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "323d7c26ddf940d0aebbf92b7dd3a77f"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "af4b0cadd4aafbbc9d7bd70bc4165b06"
server_key_iv: "125348ca307e9e53f62e52d19d2ef7d0"
server_key_version: "encryptionkey@0"
I20260619 08:54:52.500713 1779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "323d7c26ddf940d0aebbf92b7dd3a77f"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "af4b0cadd4aafbbc9d7bd70bc4165b06"
server_key_iv: "125348ca307e9e53f62e52d19d2ef7d0"
server_key_version: "encryptionkey@0"
I20260619 08:54:52.502051 1779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:52.502768 1794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:52.502935 1779 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:54:52.503033 1779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "323d7c26ddf940d0aebbf92b7dd3a77f"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "af4b0cadd4aafbbc9d7bd70bc4165b06"
server_key_iv: "125348ca307e9e53f62e52d19d2ef7d0"
server_key_version: "encryptionkey@0"
I20260619 08:54:52.503114 1779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:52.518544 1779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:52.521078 1779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:52.521243 1779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:52.525280 1779 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:45951
I20260619 08:54:52.525317 1846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:45951 every 8 connection(s)
I20260619 08:54:52.525671 1779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:54:52.526536 1847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:52.528759 1847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f: Bootstrap starting.
I20260619 08:54:52.529305 1847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:52.529583 1847 log.cc:824] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:52.530304 1847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f: No bootstrap required, opened a new log
I20260619 08:54:52.530333 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 1779
I20260619 08:54:52.530449 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:54:52.530643 16192 external_mini_cluster.cc:1428] Setting key 85612687fe80d196b751fd21ee3c712c
I20260619 08:54:52.532173 1847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323d7c26ddf940d0aebbf92b7dd3a77f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45951 } }
I20260619 08:54:52.532332 1847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:52.532397 1847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 323d7c26ddf940d0aebbf92b7dd3a77f, State: Initialized, Role: FOLLOWER
I20260619 08:54:52.532564 1847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323d7c26ddf940d0aebbf92b7dd3a77f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45951 } }
I20260619 08:54:52.532657 1847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:54:52.532702 1847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:54:52.532807 1847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:52.533703 1847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323d7c26ddf940d0aebbf92b7dd3a77f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45951 } }
I20260619 08:54:52.533823 1847 leader_election.cc:302] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 323d7c26ddf940d0aebbf92b7dd3a77f; no voters:
I20260619 08:54:52.534019 1847 leader_election.cc:288] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:54:52.534039 1852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:52.534154 1852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [term 1 LEADER]: Becoming Leader. State: Replica: 323d7c26ddf940d0aebbf92b7dd3a77f, State: Running, Role: LEADER
I20260619 08:54:52.534302 1852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323d7c26ddf940d0aebbf92b7dd3a77f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45951 } }
I20260619 08:54:52.534288 1847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:54:52.535566 1853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "323d7c26ddf940d0aebbf92b7dd3a77f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323d7c26ddf940d0aebbf92b7dd3a77f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45951 } } }
I20260619 08:54:52.535715 1853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:52.535959 1854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 323d7c26ddf940d0aebbf92b7dd3a77f. Latest consensus state: current_term: 1 leader_uuid: "323d7c26ddf940d0aebbf92b7dd3a77f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "323d7c26ddf940d0aebbf92b7dd3a77f" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 45951 } } }
I20260619 08:54:52.536118 1854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f [sys.catalog]: This master's current role is: LEADER
W20260619 08:54:52.536475 1867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:54:52.536532 1867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:54:52.536645 1868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:54:52.536861 1868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:54:52 dist-test-slave-p0h9 krb5kdc[1766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:52.542165 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:52.531132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37494 (local address 127.15.208.62:45951)
0619 08:54:52.531327 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:52.531336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:52.531365 (+ 29us) server_negotiation.cc:407] Connection header received
0619 08:54:52.532155 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:52.532161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:52.532208 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:52.532344 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:52.533179 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:52.534245 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:52.535325 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:52.536030 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:52.536146 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:52.538901 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:52.538905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:52.538905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:52.538918 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:52.540601 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:52.541004 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:52.541007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:52.541008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:52.541140 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:52.541356 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:52.541357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:52.541357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:52.541598 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:52.541786 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:52.541865 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 08:54:52.542007 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260619 08:54:52.542598 1868 catalog_manager.cc:1347] Generated new cluster ID: 8efbfd67169440aba425dcb1f25b6192
I20260619 08:54:52.542657 1868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:54:52.554680 1868 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:54:52.555225 1868 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:54:52.560783 1868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f: Generated new TSK 0
I20260619 08:54:52.560976 1868 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:54:52.591823 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:45951
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:52.689016 1873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:52.689213 1873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:52.689236 1873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:52.690455 1873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:52.690497 1873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:52.690523 1873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:54:52.691974 1873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:45951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:52.692229 1873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:52.692477 1873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:52.697504 1879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:52.697543 1878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:52.697615 1881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:52.697615 1873 server_base.cc:1034] running on GCE node
I20260619 08:54:52.697882 1873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:52.698227 1873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:52.699402 1873 hybrid_clock.cc:648] HybridClock initialized: now 1781859292699383 us; error 41 us; skew 500 ppm
Jun 19 08:54:52 dist-test-slave-p0h9 krb5kdc[1766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859292, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:52.702095 1873 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:54:52.702651 1873 webserver.cc:460] Webserver started at http://127.15.208.1:40255/ using document root <none> and password file <none>
I20260619 08:54:52.702913 1873 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:52.702991 1873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:52.703154 1873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:52.704291 1873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "ab66d899551f407296536a7a00ce62a6"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "174637734af25837e4690cc83431e4a4"
server_key_iv: "b6e8c873dea19838ca8ee4da93ae8486"
server_key_version: "encryptionkey@0"
I20260619 08:54:52.704689 1873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "ab66d899551f407296536a7a00ce62a6"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "174637734af25837e4690cc83431e4a4"
server_key_iv: "b6e8c873dea19838ca8ee4da93ae8486"
server_key_version: "encryptionkey@0"
I20260619 08:54:52.706239 1873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:52.707247 1888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:52.707471 1873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:52.707603 1873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "ab66d899551f407296536a7a00ce62a6"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "174637734af25837e4690cc83431e4a4"
server_key_iv: "b6e8c873dea19838ca8ee4da93ae8486"
server_key_version: "encryptionkey@0"
I20260619 08:54:52.707680 1873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:52.716470 1873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:52.718717 1873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:52.718894 1873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:52.719136 1873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:52.719462 1873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:52.719520 1873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:52.719592 1873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:52.719615 1873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:52.725845 1873 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:45269
I20260619 08:54:52.725893 2001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:45269 every 8 connection(s)
I20260619 08:54:52.726210 1873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:54:52.727702 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 1873
I20260619 08:54:52.727829 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:54:52.728006 16192 external_mini_cluster.cc:1428] Setting key 3d6c1d5960d8721dce4326e21e1bce8e
Jun 19 08:54:52 dist-test-slave-p0h9 krb5kdc[1766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859292, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:52.737739 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:52.726891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:56113 (local address 127.15.208.62:45951)
0619 08:54:52.727007 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:52.727009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:52.727203 (+ 194us) server_negotiation.cc:407] Connection header received
0619 08:54:52.727718 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:52.727720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:52.727737 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:52.727823 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:52.729065 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:52.729733 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:52.730809 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:52.731150 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:52.731193 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:52.734148 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:52.734150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:52.734151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:52.734161 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:52.735992 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:52.736598 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:52.736599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:52.736600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:52.736663 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:52.736990 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:52.736992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:52.736992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:52.737129 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:52.737214 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:52.737550 (+ 336us) server_negotiation.cc:299] Negotiation successful
0619 08:54:52.737647 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
I20260619 08:54:52.737938 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:52.726871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:45951 (local address 127.15.208.1:56113)
0619 08:54:52.727073 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:52.727109 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:52.727594 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:52.727927 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:52.727932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:52.728635 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:52.728948 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:52.728954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:52.729831 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:52.729831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:52.730697 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:52.730700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:52.731286 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:52.731287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:52.731443 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:52.732086 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:52.732096 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:52.734033 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:52.736088 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:52.736095 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:52.736098 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:52.736472 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:52.736771 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:52.736772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:52.736772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:52.736926 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:52.737204 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:52.737210 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:52.737361 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 08:54:52.737515 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 08:54:52.737682 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":117,"threads_started":1}
I20260619 08:54:52.738435 2002 heartbeater.cc:347] Connected to a master server at 127.15.208.62:45951
I20260619 08:54:52.738678 2002 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:52.738988 2002 heartbeater.cc:511] Master 127.15.208.62:45951 requested a full tablet report, sending...
I20260619 08:54:52.739537 1811 ts_manager.cc:194] Registered new tserver with Master: ab66d899551f407296536a7a00ce62a6 (127.15.208.1:45269)
I20260619 08:54:52.740306 1811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:56113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:54:52.755661 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:45951
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:52.836583 2007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:52.836805 2007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:52.836829 2007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:52.837967 2007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:52.838013 2007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:52.838039 2007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:54:52.839560 2007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:45951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:52.839798 2007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:52.840025 2007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:52.845072 2012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:52.845072 2015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:52.845201 2013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:52.845580 2007 server_base.cc:1034] running on GCE node
I20260619 08:54:52.845770 2007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:52.845975 2007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:52.847119 2007 hybrid_clock.cc:648] HybridClock initialized: now 1781859292847110 us; error 32 us; skew 500 ppm
Jun 19 08:54:52 dist-test-slave-p0h9 krb5kdc[1766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859292, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:52.849912 2007 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:54:52.850387 2007 webserver.cc:460] Webserver started at http://127.15.208.2:33595/ using document root <none> and password file <none>
I20260619 08:54:52.850651 2007 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:52.850718 2007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:52.850857 2007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:52.851914 2007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "685fe43be8d3492b81b2d0ca34df93d6"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "a15d985ac78a5d7286f9f14149861c08"
server_key_iv: "405431fb17add0676c016a7dd0448bf3"
server_key_version: "encryptionkey@0"
I20260619 08:54:52.852270 2007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "685fe43be8d3492b81b2d0ca34df93d6"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "a15d985ac78a5d7286f9f14149861c08"
server_key_iv: "405431fb17add0676c016a7dd0448bf3"
server_key_version: "encryptionkey@0"
I20260619 08:54:52.853719 2007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:54:52.854588 2022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:52.854785 2007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:52.854877 2007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "685fe43be8d3492b81b2d0ca34df93d6"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "a15d985ac78a5d7286f9f14149861c08"
server_key_iv: "405431fb17add0676c016a7dd0448bf3"
server_key_version: "encryptionkey@0"
I20260619 08:54:52.854960 2007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:52.862864 2007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:52.864848 2007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:52.864998 2007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:52.865227 2007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:52.865527 2007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:52.865578 2007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:52.865618 2007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:52.865645 2007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:52.871479 2007 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:42005
I20260619 08:54:52.871536 2135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:42005 every 8 connection(s)
I20260619 08:54:52.871874 2007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:54:52 dist-test-slave-p0h9 krb5kdc[1766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859292, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:52.880779 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 2007
I20260619 08:54:52.880921 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:54:52.880936 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:52.872652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:52155 (local address 127.15.208.62:45951)
0619 08:54:52.872700 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:52.872701 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:52.872957 (+ 256us) server_negotiation.cc:407] Connection header received
0619 08:54:52.873475 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:52.873476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:52.873488 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:52.873545 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:52.874476 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:52.874990 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:52.875999 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:52.876210 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:52.876297 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:52.878372 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:52.878373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:52.878373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:52.878380 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:52.879622 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:52.879986 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:52.879987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:52.879987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:52.880035 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:52.880244 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:52.880244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:52.880245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:52.880340 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:52.880389 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:52.880682 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 08:54:52.880820 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:54:52.881115 2138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:52.872613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:45951 (local address 127.15.208.2:52155)
0619 08:54:52.872809 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:52.872855 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:52.873366 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:52.873616 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:52.873620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:52.874188 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:52.874368 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:52.874373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:52.875081 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:52.875082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:52.875882 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:52.875884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:52.876307 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:52.876308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:52.876433 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:52.876960 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:52.876968 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:52.878286 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:52.879704 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:52.879710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:52.879711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:52.879921 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:52.880092 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:52.880092 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:52.880093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:52.880190 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:52.880395 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:52.880399 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:52.880505 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 08:54:52.880666 (+ 161us) client_negotiation.cc:240] Negotiation successful
0619 08:54:52.880867 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":133,"threads_started":1}
I20260619 08:54:52.881140 16192 external_mini_cluster.cc:1428] Setting key 8b77b270eda07758acd3db6b63ac3622
I20260619 08:54:52.881613 2136 heartbeater.cc:347] Connected to a master server at 127.15.208.62:45951
I20260619 08:54:52.881719 2136 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:52.881906 2136 heartbeater.cc:511] Master 127.15.208.62:45951 requested a full tablet report, sending...
I20260619 08:54:52.882330 1810 ts_manager.cc:194] Registered new tserver with Master: 685fe43be8d3492b81b2d0ca34df93d6 (127.15.208.2:42005)
I20260619 08:54:52.882706 1810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:52155
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:54:52.905004 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:45951
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:52.985481 2141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:52.985687 2141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:52.985710 2141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:52.986874 2141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:52.986922 2141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:52.986949 2141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:54:52.988474 2141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:45951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:52.988735 2141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:52.989032 2141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:52.994028 2146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:52.994053 2149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:52.994091 2141 server_base.cc:1034] running on GCE node
W20260619 08:54:52.994232 2147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:52.994458 2141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:52.994752 2141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:52.995913 2141 hybrid_clock.cc:648] HybridClock initialized: now 1781859292995896 us; error 48 us; skew 500 ppm
Jun 19 08:54:52 dist-test-slave-p0h9 krb5kdc[1766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859292, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:52.998507 2141 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:54:52.998968 2141 webserver.cc:460] Webserver started at http://127.15.208.3:41545/ using document root <none> and password file <none>
I20260619 08:54:52.999171 2141 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:52.999212 2141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:52.999300 2141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:53.000352 2141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "69d6bee89df24c53a30056754f3a32b1"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "59e6e93a197f1a67271c543a79315975"
server_key_iv: "cc2b2642d90f879febdeabd37d9c21e6"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.000712 2141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "69d6bee89df24c53a30056754f3a32b1"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "59e6e93a197f1a67271c543a79315975"
server_key_iv: "cc2b2642d90f879febdeabd37d9c21e6"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.002094 2141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:53.002866 2156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.003037 2141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:53.003123 2141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "69d6bee89df24c53a30056754f3a32b1"
format_stamp: "Formatted at 2026-06-19 08:54:52 on dist-test-slave-p0h9"
server_key: "59e6e93a197f1a67271c543a79315975"
server_key_iv: "cc2b2642d90f879febdeabd37d9c21e6"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.003185 2141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:53.013008 2141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:53.014966 2141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:53.015096 2141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:53.015280 2141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:53.015600 2141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:53.015636 2141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.015676 2141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:53.015704 2141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.021481 2141 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:38041
I20260619 08:54:53.021572 2269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:38041 every 8 connection(s)
I20260619 08:54:53.021878 2141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:54:53 dist-test-slave-p0h9 krb5kdc[1766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859292, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:53.029955 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 2141
I20260619 08:54:53.030105 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:54:53.030325 16192 external_mini_cluster.cc:1428] Setting key 73ccc3103355304d0d367e10531b735f
I20260619 08:54:53.031739 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.022611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:33847 (local address 127.15.208.62:45951)
0619 08:54:53.022654 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.022655 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.022912 (+ 257us) server_negotiation.cc:407] Connection header received
0619 08:54:53.023921 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.023923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.023935 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.023994 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:53.024640 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.025224 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.026264 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.026461 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.026549 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.028654 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:53.028656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:53.028656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:53.028663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:53.030179 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:53.030645 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:53.030646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:53.030646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:53.030714 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:53.031076 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:53.031077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:53.031077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:53.031208 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:53.031299 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.031604 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.031676 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:54:53.031939 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.022579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:45951 (local address 127.15.208.3:33847)
0619 08:54:53.022773 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.022829 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.023803 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.024068 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.024072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.024341 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:53.024531 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.024536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.025324 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.025325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.026154 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.026156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.026566 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.026567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.026700 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.027163 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:53.027171 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:53.028560 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:53.030284 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:53.030292 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:53.030294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:53.030584 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:53.030821 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:53.030822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:53.030822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:53.030986 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:53.031301 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:53.031307 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:53.031473 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.031563 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.031694 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":105,"threads_started":1}
I20260619 08:54:53.032433 2270 heartbeater.cc:347] Connected to a master server at 127.15.208.62:45951
I20260619 08:54:53.032536 2270 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:53.032716 2270 heartbeater.cc:511] Master 127.15.208.62:45951 requested a full tablet report, sending...
I20260619 08:54:53.033169 1810 ts_manager.cc:194] Registered new tserver with Master: 69d6bee89df24c53a30056754f3a32b1 (127.15.208.3:38041)
I20260619 08:54:53.033542 1810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:33847
I20260619 08:54:53.042357 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:54:53.049899 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.043700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37496 (local address 127.15.208.62:45951)
0619 08:54:53.043757 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.043758 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.043901 (+ 143us) server_negotiation.cc:407] Connection header received
0619 08:54:53.043978 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.043979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.043990 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.044082 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:53.044639 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.045205 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.045952 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.046152 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.046229 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.047491 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:53.047493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:53.047493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:53.047500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:53.048609 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:53.049027 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:53.049029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:53.049029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:53.049102 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:53.049339 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:53.049340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:53.049341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:53.049477 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:53.049593 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.049726 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.049802 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:54:53.051416 1810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:54:53.052968 1810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:54:53.059448 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.055060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:45269 (local address 127.0.0.1:44358)
0619 08:54:53.055212 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.055257 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.055325 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.055789 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.055792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.055794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:53.055861 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.055864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.057299 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.057300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.058382 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.058384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.059132 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.059133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.059233 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.059247 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.059324 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.059374 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260619 08:54:53.059486 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.055171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44358 (local address 127.15.208.1:45269)
0619 08:54:53.055594 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.055598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.055619 (+ 21us) server_negotiation.cc:407] Connection header received
0619 08:54:53.055662 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.055664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.055678 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.055769 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:53.056030 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.057189 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.058488 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.059020 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.059111 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.059174 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.059358 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.059405 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":104,"threads_started":1}
I20260619 08:54:53.060331 1936 tablet_service.cc:1467] Processing CreateTablet for tablet 0c0541470411436789b786a821eb69c0 (DEFAULT_TABLE table=test-table [id=0e9d3630b5f841bc8f23309524fe975c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:53.060467 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.055435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:42005 (local address 127.0.0.1:57064)
0619 08:54:53.055582 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.055593 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.055646 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.055961 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.055963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.055966 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:53.056014 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.056016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.058532 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.058533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.059270 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.059272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.060223 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.060224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.060294 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.060298 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.060348 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.060388 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":52,"threads_started":1}
I20260619 08:54:53.060582 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.055394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37908 (local address 127.15.208.3:38041)
0619 08:54:53.055879 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.055882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.055892 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:53.055928 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.055930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.055946 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.056015 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:53.056166 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.056937 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.059212 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.059986 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.060057 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.060116 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.060461 (+ 345us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.060506 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":92,"threads_started":1}
I20260619 08:54:53.060781 1936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c0541470411436789b786a821eb69c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:53.060576 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.055296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:38041 (local address 127.0.0.1:37908)
0619 08:54:53.055520 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.055531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.055585 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.056077 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.056078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.056080 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:53.056111 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.056112 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.057035 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.057036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.058382 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.058384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.060152 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.060153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.060372 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.060375 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.060427 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.060460 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":24,"thread_start_us":51,"threads_started":1}
I20260619 08:54:53.060911 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.055542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57064 (local address 127.15.208.2:42005)
0619 08:54:53.055732 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.055736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.055746 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:53.055787 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.055789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.055805 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.055882 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:53.056090 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.057200 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.059382 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.060142 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.060223 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.060276 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.060787 (+ 511us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.060836 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":81,"threads_started":1}
I20260619 08:54:53.061589 2070 tablet_service.cc:1467] Processing CreateTablet for tablet 0c0541470411436789b786a821eb69c0 (DEFAULT_TABLE table=test-table [id=0e9d3630b5f841bc8f23309524fe975c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:53.061697 2204 tablet_service.cc:1467] Processing CreateTablet for tablet 0c0541470411436789b786a821eb69c0 (DEFAULT_TABLE table=test-table [id=0e9d3630b5f841bc8f23309524fe975c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:53.061941 2070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c0541470411436789b786a821eb69c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:53.061951 2204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c0541470411436789b786a821eb69c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:53.063649 2284 tablet_bootstrap.cc:492] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6: Bootstrap starting.
I20260619 08:54:53.064303 2284 tablet_bootstrap.cc:654] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:53.064905 2284 log.cc:824] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:53.065186 2285 tablet_bootstrap.cc:492] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: Bootstrap starting.
I20260619 08:54:53.065532 2287 tablet_bootstrap.cc:492] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: Bootstrap starting.
I20260619 08:54:53.065863 2285 tablet_bootstrap.cc:654] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:53.066308 2287 tablet_bootstrap.cc:654] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:53.066428 2284 tablet_bootstrap.cc:492] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6: No bootstrap required, opened a new log
I20260619 08:54:53.066478 2285 log.cc:824] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:53.066521 2284 ts_tablet_manager.cc:1397] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 08:54:53.066877 2287 log.cc:824] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:53.067296 2285 tablet_bootstrap.cc:492] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: No bootstrap required, opened a new log
I20260619 08:54:53.067414 2285 ts_tablet_manager.cc:1397] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:54:53.067644 2287 tablet_bootstrap.cc:492] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: No bootstrap required, opened a new log
I20260619 08:54:53.067711 2287 ts_tablet_manager.cc:1397] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:54:53.068361 2284 raft_consensus.cc:348] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 } } peers { permanent_uuid: "69d6bee89df24c53a30056754f3a32b1" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38041 } } peers { permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 } }
I20260619 08:54:53.068526 2284 raft_consensus.cc:374] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:53.068569 2284 raft_consensus.cc:729] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab66d899551f407296536a7a00ce62a6, State: Initialized, Role: FOLLOWER
I20260619 08:54:53.068682 2284 consensus_queue.cc:260] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 } } peers { permanent_uuid: "69d6bee89df24c53a30056754f3a32b1" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38041 } } peers { permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 } }
I20260619 08:54:53.068977 2284 ts_tablet_manager.cc:1428] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260619 08:54:53.069137 2002 heartbeater.cc:503] Master 127.15.208.62:45951 was elected leader, sending a full tablet report...
I20260619 08:54:53.069433 2285 raft_consensus.cc:348] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 } } peers { permanent_uuid: "69d6bee89df24c53a30056754f3a32b1" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38041 } } peers { permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 } }
I20260619 08:54:53.069617 2285 raft_consensus.cc:374] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:53.069651 2285 raft_consensus.cc:729] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69d6bee89df24c53a30056754f3a32b1, State: Initialized, Role: FOLLOWER
I20260619 08:54:53.069763 2285 consensus_queue.cc:260] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 } } peers { permanent_uuid: "69d6bee89df24c53a30056754f3a32b1" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38041 } } peers { permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 } }
I20260619 08:54:53.070050 2285 ts_tablet_manager.cc:1428] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:54:53.070058 2270 heartbeater.cc:503] Master 127.15.208.62:45951 was elected leader, sending a full tablet report...
I20260619 08:54:53.070218 2287 raft_consensus.cc:348] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 } } peers { permanent_uuid: "69d6bee89df24c53a30056754f3a32b1" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38041 } } peers { permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 } }
I20260619 08:54:53.070363 2287 raft_consensus.cc:374] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:53.070402 2287 raft_consensus.cc:729] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 685fe43be8d3492b81b2d0ca34df93d6, State: Initialized, Role: FOLLOWER
I20260619 08:54:53.070508 2287 consensus_queue.cc:260] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 } } peers { permanent_uuid: "69d6bee89df24c53a30056754f3a32b1" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38041 } } peers { permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 } }
I20260619 08:54:53.070799 2287 ts_tablet_manager.cc:1428] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:54:53.070829 2136 heartbeater.cc:503] Master 127.15.208.62:45951 was elected leader, sending a full tablet report...
I20260619 08:54:53.110242 2291 raft_consensus.cc:482] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:53.110363 2291 raft_consensus.cc:504] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 } } peers { permanent_uuid: "69d6bee89df24c53a30056754f3a32b1" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38041 } } peers { permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 } }
I20260619 08:54:53.110775 2291 leader_election.cc:288] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab66d899551f407296536a7a00ce62a6 (127.15.208.1:45269), 685fe43be8d3492b81b2d0ca34df93d6 (127.15.208.2:42005)
I20260619 08:54:53.113924 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.111045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:34789 (local address 127.15.208.1:45269)
0619 08:54:53.111110 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.111112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.111159 (+ 47us) server_negotiation.cc:407] Connection header received
0619 08:54:53.111191 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.111192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.111203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.111274 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:53.111481 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.112069 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.113127 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.113544 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.113603 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.113612 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.113801 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.113848 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:54:53.113940 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.111233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:60579 (local address 127.15.208.2:42005)
0619 08:54:53.111285 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.111286 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.111428 (+ 142us) server_negotiation.cc:407] Connection header received
0619 08:54:53.111481 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.111482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.111500 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.111543 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:53.111725 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.112223 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.113211 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.113598 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.113673 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.113681 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.113834 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.113875 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 08:54:53.113938 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.110985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:45269 (local address 127.15.208.3:34789)
0619 08:54:53.111042 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.111054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.111102 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.111295 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.111297 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.111299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:53.111375 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.111377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.112147 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.112148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.113018 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.113020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.113634 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.113635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.113715 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.113718 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.113770 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.113827 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:54:53.113946 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.111144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:42005 (local address 127.15.208.3:60579)
0619 08:54:53.111351 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.111359 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.111409 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.111572 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.111573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.111575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:53.111655 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.111656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.112301 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.112301 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.113125 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.113127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.113685 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.113686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.113740 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.113742 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.113818 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.113852 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":129,"threads_started":1}
I20260619 08:54:53.114408 1956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0541470411436789b786a821eb69c0" candidate_uuid: "69d6bee89df24c53a30056754f3a32b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab66d899551f407296536a7a00ce62a6" is_pre_election: true
I20260619 08:54:53.114600 1956 raft_consensus.cc:2454] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69d6bee89df24c53a30056754f3a32b1 in term 0.
I20260619 08:54:53.114645 2090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0541470411436789b786a821eb69c0" candidate_uuid: "69d6bee89df24c53a30056754f3a32b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "685fe43be8d3492b81b2d0ca34df93d6" is_pre_election: true
I20260619 08:54:53.114789 2090 raft_consensus.cc:2454] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69d6bee89df24c53a30056754f3a32b1 in term 0.
I20260619 08:54:53.114845 2158 leader_election.cc:302] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69d6bee89df24c53a30056754f3a32b1, ab66d899551f407296536a7a00ce62a6; no voters:
I20260619 08:54:53.115005 2291 raft_consensus.cc:2781] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:54:53.115064 2291 raft_consensus.cc:482] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:54:53.115093 2291 raft_consensus.cc:3037] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:53.115852 2291 raft_consensus.cc:504] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 } } peers { permanent_uuid: "69d6bee89df24c53a30056754f3a32b1" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38041 } } peers { permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 } }
I20260619 08:54:53.115981 2291 leader_election.cc:288] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [CANDIDATE]: Term 1 election: Requested vote from peers ab66d899551f407296536a7a00ce62a6 (127.15.208.1:45269), 685fe43be8d3492b81b2d0ca34df93d6 (127.15.208.2:42005)
I20260619 08:54:53.116181 2090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0541470411436789b786a821eb69c0" candidate_uuid: "69d6bee89df24c53a30056754f3a32b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "685fe43be8d3492b81b2d0ca34df93d6"
I20260619 08:54:53.116215 1956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0541470411436789b786a821eb69c0" candidate_uuid: "69d6bee89df24c53a30056754f3a32b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab66d899551f407296536a7a00ce62a6"
I20260619 08:54:53.116259 2090 raft_consensus.cc:3037] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:53.116324 1956 raft_consensus.cc:3037] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:53.117095 2090 raft_consensus.cc:2454] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69d6bee89df24c53a30056754f3a32b1 in term 1.
I20260619 08:54:53.117262 2158 leader_election.cc:302] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 685fe43be8d3492b81b2d0ca34df93d6, 69d6bee89df24c53a30056754f3a32b1; no voters:
I20260619 08:54:53.117268 1956 raft_consensus.cc:2454] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69d6bee89df24c53a30056754f3a32b1 in term 1.
I20260619 08:54:53.117375 2291 raft_consensus.cc:2781] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:53.117540 2291 raft_consensus.cc:686] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 1 LEADER]: Becoming Leader. State: Replica: 69d6bee89df24c53a30056754f3a32b1, State: Running, Role: LEADER
I20260619 08:54:53.117616 2291 consensus_queue.cc:237] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 } } peers { permanent_uuid: "69d6bee89df24c53a30056754f3a32b1" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38041 } } peers { permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 } }
I20260619 08:54:53.118462 1811 catalog_manager.cc:5495] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69d6bee89df24c53a30056754f3a32b1 (127.15.208.3). New cstate: current_term: 1 leader_uuid: "69d6bee89df24c53a30056754f3a32b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69d6bee89df24c53a30056754f3a32b1" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 38041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 } health_report { overall_health: UNKNOWN } } }
W20260619 08:54:53.122398 2137 tablet.cc:2369] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:54:53.135136 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.132535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37924 (local address 127.15.208.3:38041)
0619 08:54:53.132605 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.132608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.132636 (+ 28us) server_negotiation.cc:407] Connection header received
0619 08:54:53.132680 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.132681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.132698 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.132817 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:54:53.132958 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.133426 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.134217 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.134464 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.134541 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.134702 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:54:53.134848 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:54:53.134986 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.135018 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.135029 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:54:53.137064 2090 raft_consensus.cc:1261] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6 [term 1 FOLLOWER]: Refusing update from remote peer 69d6bee89df24c53a30056754f3a32b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:53.137082 1956 raft_consensus.cc:1261] T 0c0541470411436789b786a821eb69c0 P ab66d899551f407296536a7a00ce62a6 [term 1 FOLLOWER]: Refusing update from remote peer 69d6bee89df24c53a30056754f3a32b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:53.137457 2291 consensus_queue.cc:1035] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "685fe43be8d3492b81b2d0ca34df93d6" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:53.137627 2294 consensus_queue.cc:1035] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab66d899551f407296536a7a00ce62a6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 45269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:53.139787 2297 mvcc.cc:205] Tried to move back new op lower bound from 7298495664686743552 to 7298495664610500608. Current Snapshot: MvccSnapshot[applied={T|T < 7298495664686743552}]
I20260619 08:54:53.140290 2298 mvcc.cc:205] Tried to move back new op lower bound from 7298495664686743552 to 7298495664610500608. Current Snapshot: MvccSnapshot[applied={T|T < 7298495664686743552}]
I20260619 08:54:53.142380 1811 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:53.142494 1811 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:53.143276 1811 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 323d7c26ddf940d0aebbf92b7dd3a77f: Sending DeleteTablet for 3 replicas of tablet 0c0541470411436789b786a821eb69c0
I20260619 08:54:53.143676 2204 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c0541470411436789b786a821eb69c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:53 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:37908
I20260619 08:54:53.143685 1936 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c0541470411436789b786a821eb69c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:53 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:44358
I20260619 08:54:53.143728 2070 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c0541470411436789b786a821eb69c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:53 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:57064
I20260619 08:54:53.144055 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 1873
I20260619 08:54:53.148859 2309 tablet_replica.cc:330] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: stopping tablet replica
I20260619 08:54:53.148996 2309 raft_consensus.cc:2229] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:53.149129 2309 raft_consensus.cc:2258] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:53.149138 2308 tablet_replica.cc:330] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: stopping tablet replica
I20260619 08:54:53.149322 2308 raft_consensus.cc:2229] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:54:53.149459 2308 raft_consensus.cc:2258] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:53.149868 2309 ts_tablet_manager.cc:1905] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:53.151423 2309 ts_tablet_manager.cc:1918] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:53.151573 2309 log.cc:1196] T 0c0541470411436789b786a821eb69c0 P 685fe43be8d3492b81b2d0ca34df93d6: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/0c0541470411436789b786a821eb69c0
W20260619 08:54:53.151607 1796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:45269 (error 108)
I20260619 08:54:53.151485 2308 ts_tablet_manager.cc:1905] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:53.151867 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 2007
I20260619 08:54:53.153805 2308 ts_tablet_manager.cc:1918] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:53.153877 2308 log.cc:1196] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/0c0541470411436789b786a821eb69c0
I20260619 08:54:53.154145 2308 ts_tablet_manager.cc:1939] T 0c0541470411436789b786a821eb69c0 P 69d6bee89df24c53a30056754f3a32b1: Deleting consensus metadata
I20260619 08:54:53.154558 1796 catalog_manager.cc:4841] TS 69d6bee89df24c53a30056754f3a32b1 (127.15.208.3:38041): tablet 0c0541470411436789b786a821eb69c0 (table test-table [id=0e9d3630b5f841bc8f23309524fe975c]) successfully deleted
I20260619 08:54:53.156090 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.153847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:45269 (local address 127.0.0.1:44372)
0619 08:54:53.156008 (+ 2161us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.156060 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:45269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2139}
W20260619 08:54:53.156288 1796 catalog_manager.cc:4584] TS ab66d899551f407296536a7a00ce62a6 (127.15.208.1:45269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c0541470411436789b786a821eb69c0: Network error: Client connection negotiation failed: client connection to 127.15.208.1:45269: connect: Connection refused (error 111)
I20260619 08:54:53.158195 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.158082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:42005 (local address 127.0.0.1:57068)
0619 08:54:53.158140 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.158171 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.2:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 08:54:53.158188 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 2141
I20260619 08:54:53.162793 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 1779
2026-06-19T08:54:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1908 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T08:54:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:54:53Z Disabled control of system clock
I20260619 08:54:53.189952 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46575
--webserver_interface=127.15.208.62
--webserver_port=0
--builtin_ntp_servers=127.15.208.20:40123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:46575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:54:53.269944 2314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:53.270165 2314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:53.270186 2314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:53.271235 2314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:54:53.271270 2314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:53.271283 2314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:53.271296 2314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:54:53.271308 2314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:54:53.272536 2314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:46575
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:53.272809 2314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:53.273064 2314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:53.277796 2320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:53.277864 2319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:53.277936 2314 server_base.cc:1034] running on GCE node
W20260619 08:54:53.278007 2322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:53.278295 2314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:53.278630 2314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:53.279790 2314 hybrid_clock.cc:648] HybridClock initialized: now 1781859293279771 us; error 33 us; skew 500 ppm
I20260619 08:54:53.281049 2314 webserver.cc:460] Webserver started at http://127.15.208.62:43483/ using document root <none> and password file <none>
I20260619 08:54:53.281298 2314 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:53.281366 2314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:53.281520 2314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:53.282630 2314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "10da46d3ecdd420fbac697a25ba08ded"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "fb3063ef615dad44142e4fc306f06fca"
server_key_iv: "26a6cbfacd0ff1f0c79088d09cef4521"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.283047 2314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "10da46d3ecdd420fbac697a25ba08ded"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "fb3063ef615dad44142e4fc306f06fca"
server_key_iv: "26a6cbfacd0ff1f0c79088d09cef4521"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.284340 2314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:54:53.285169 2328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.285349 2314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.285440 2314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "10da46d3ecdd420fbac697a25ba08ded"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "fb3063ef615dad44142e4fc306f06fca"
server_key_iv: "26a6cbfacd0ff1f0c79088d09cef4521"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.285521 2314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:53.294039 2314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:53.294348 2314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:53.294497 2314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:53.298659 2314 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:46575
I20260619 08:54:53.298691 2380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:46575 every 8 connection(s)
I20260619 08:54:53.299052 2314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:54:53.300034 2381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:53.302526 2381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded: Bootstrap starting.
I20260619 08:54:53.303133 2381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:53.303464 2381 log.cc:824] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:53.304116 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 2314
I20260619 08:54:53.304243 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:54:53.304311 2381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded: No bootstrap required, opened a new log
I20260619 08:54:53.304481 16192 external_mini_cluster.cc:1428] Setting key d11a49c54b77876e3e0465e92cda45e0
I20260619 08:54:53.306247 2381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10da46d3ecdd420fbac697a25ba08ded" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46575 } }
I20260619 08:54:53.306407 2381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:53.306473 2381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10da46d3ecdd420fbac697a25ba08ded, State: Initialized, Role: FOLLOWER
I20260619 08:54:53.306653 2381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10da46d3ecdd420fbac697a25ba08ded" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46575 } }
I20260619 08:54:53.306774 2381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:54:53.306819 2381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:54:53.306892 2381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:53.307518 2381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10da46d3ecdd420fbac697a25ba08ded" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46575 } }
I20260619 08:54:53.307603 2381 leader_election.cc:302] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10da46d3ecdd420fbac697a25ba08ded; no voters:
I20260619 08:54:53.307790 2381 leader_election.cc:288] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:54:53.307821 2386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:53.307998 2386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [term 1 LEADER]: Becoming Leader. State: Replica: 10da46d3ecdd420fbac697a25ba08ded, State: Running, Role: LEADER
I20260619 08:54:53.308063 2381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:54:53.308130 2386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10da46d3ecdd420fbac697a25ba08ded" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46575 } }
I20260619 08:54:53.308516 2386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10da46d3ecdd420fbac697a25ba08ded. Latest consensus state: current_term: 1 leader_uuid: "10da46d3ecdd420fbac697a25ba08ded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10da46d3ecdd420fbac697a25ba08ded" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46575 } } }
I20260619 08:54:53.308627 2386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:53.308552 2387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10da46d3ecdd420fbac697a25ba08ded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10da46d3ecdd420fbac697a25ba08ded" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46575 } } }
I20260619 08:54:53.308838 2387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded [sys.catalog]: This master's current role is: LEADER
W20260619 08:54:53.310245 2402 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:54:53.310318 2402 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:54:53.310616 2392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:54:53.310851 2392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:54:53.311842 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.305081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39666 (local address 127.15.208.62:46575)
0619 08:54:53.305325 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.305332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.305356 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:54:53.306238 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.306246 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.306310 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.306485 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:53.306728 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.308417 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.309606 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.310250 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.310370 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.311162 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:53.311167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:53.311167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:53.311357 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:53.311392 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:53.311403 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:53.311488 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.311581 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.311663 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260619 08:54:53.312681 2392 catalog_manager.cc:1347] Generated new cluster ID: 8a84e9a8e24d4e5788769d8b18934407
I20260619 08:54:53.312769 2392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:54:53.321748 2392 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:54:53.322275 2392 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:54:53.327313 2392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded: Generated new TSK 0
I20260619 08:54:53.327523 2392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:54:53.333051 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46575
--builtin_ntp_servers=127.15.208.20:40123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:54:53.434523 2405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:53.434733 2405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:53.434756 2405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:53.435928 2405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:53.435972 2405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:53.436002 2405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:54:53.437422 2405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:53.437668 2405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:53.437983 2405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:53.443866 2410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:53.444000 2411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:53.444017 2413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:53.444154 2405 server_base.cc:1034] running on GCE node
I20260619 08:54:53.444427 2405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:53.444698 2405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:53.445881 2405 hybrid_clock.cc:648] HybridClock initialized: now 1781859293445868 us; error 30 us; skew 500 ppm
I20260619 08:54:53.447201 2405 webserver.cc:460] Webserver started at http://127.15.208.1:41063/ using document root <none> and password file <none>
I20260619 08:54:53.447435 2405 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:53.447511 2405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:53.447667 2405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:53.448797 2405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "e7f22218da8f45adbdc52587d1a05baa"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "8e383a81a39a79c422659cdfab49b891"
server_key_iv: "7e5c914fe183a2cb532c7a8ea6425ffb"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.449173 2405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "e7f22218da8f45adbdc52587d1a05baa"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "8e383a81a39a79c422659cdfab49b891"
server_key_iv: "7e5c914fe183a2cb532c7a8ea6425ffb"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.450491 2405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:53.451284 2419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.451436 2405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:54:53.451530 2405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "e7f22218da8f45adbdc52587d1a05baa"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "8e383a81a39a79c422659cdfab49b891"
server_key_iv: "7e5c914fe183a2cb532c7a8ea6425ffb"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.451596 2405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:53.461596 2405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:53.461879 2405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:53.462023 2405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:53.462268 2405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:53.462667 2405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:53.462704 2405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.462725 2405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:53.462738 2405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.469123 2405 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:42459
I20260619 08:54:53.469201 2532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:42459 every 8 connection(s)
I20260619 08:54:53.469539 2405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:54:53.475126 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.470312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:52611 (local address 127.15.208.62:46575)
0619 08:54:53.470372 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.470374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.470637 (+ 263us) server_negotiation.cc:407] Connection header received
0619 08:54:53.471203 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.471204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.471218 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.471273 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:53.472013 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.472515 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.473594 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.473802 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.473885 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.474792 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:53.474793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:53.474793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:53.474801 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:53.474817 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:53.474822 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:53.474894 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.475012 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.475045 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:54:53.475343 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.470311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46575 (local address 127.15.208.1:52611)
0619 08:54:53.470514 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.470548 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.471093 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.471358 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.471361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.471369 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:53.471893 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.471899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.472619 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.472620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.473475 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.473478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.473895 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.473896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.474038 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.474628 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:54:53.474629 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:53.474660 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:54:53.474683 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:53.474893 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:53.474900 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:53.474903 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.475005 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.475108 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260619 08:54:53.475889 2533 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46575
I20260619 08:54:53.475975 2533 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:53.476179 2533 heartbeater.cc:511] Master 127.15.208.62:46575 requested a full tablet report, sending...
I20260619 08:54:53.476706 2345 ts_manager.cc:194] Registered new tserver with Master: e7f22218da8f45adbdc52587d1a05baa (127.15.208.1:42459)
I20260619 08:54:53.477389 2345 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.1:52611
I20260619 08:54:53.478858 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 2405
I20260619 08:54:53.478979 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:54:53.479185 16192 external_mini_cluster.cc:1428] Setting key a41210ab89b053ee084fb6f5816392bb
I20260619 08:54:53.480123 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46575
--builtin_ntp_servers=127.15.208.20:40123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:54:53.560537 2536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:53.560807 2536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:53.560858 2536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:53.562037 2536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:53.562078 2536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:53.562105 2536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:54:53.563483 2536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:53.563737 2536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:53.564044 2536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:53.569041 2542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:53.569092 2536 server_base.cc:1034] running on GCE node
W20260619 08:54:53.569046 2544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:53.569057 2541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:53.569465 2536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:53.569749 2536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:53.570927 2536 hybrid_clock.cc:648] HybridClock initialized: now 1781859293570908 us; error 31 us; skew 500 ppm
I20260619 08:54:53.572609 2536 webserver.cc:460] Webserver started at http://127.15.208.2:37753/ using document root <none> and password file <none>
I20260619 08:54:53.572961 2536 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:53.573046 2536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:53.573195 2536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:53.574471 2536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "589951a9b2c44526b6485191ea9a04f9"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "99d812ba9dafe0f0411731c76b5aeca0"
server_key_iv: "e218ea9a391ade9e33a4729d9da3db77"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.574846 2536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "589951a9b2c44526b6485191ea9a04f9"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "99d812ba9dafe0f0411731c76b5aeca0"
server_key_iv: "e218ea9a391ade9e33a4729d9da3db77"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.576148 2536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:54:53.576908 2550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.577085 2536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:54:53.577183 2536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "589951a9b2c44526b6485191ea9a04f9"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "99d812ba9dafe0f0411731c76b5aeca0"
server_key_iv: "e218ea9a391ade9e33a4729d9da3db77"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.577281 2536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:53.620409 2536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:53.620828 2536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:53.621026 2536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:53.621311 2536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:53.621685 2536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:53.621740 2536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.621785 2536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:53.621812 2536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.627589 2536 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:46819
I20260619 08:54:53.627672 2663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:46819 every 8 connection(s)
I20260619 08:54:53.628013 2536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:54:53.633450 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.628804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:53941 (local address 127.15.208.62:46575)
0619 08:54:53.628848 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.628850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.629061 (+ 211us) server_negotiation.cc:407] Connection header received
0619 08:54:53.629616 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.629617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.629629 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.629688 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:53.630426 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.630948 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.631980 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.632197 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.632286 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.633157 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:53.633159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:53.633159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:53.633164 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:53.633180 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:53.633184 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:53.633239 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.633340 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.633370 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:54:53.633713 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.628768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46575 (local address 127.15.208.2:53941)
0619 08:54:53.628941 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.628975 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.629476 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.629754 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.629758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.629768 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:53.630318 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.630323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.631068 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.631069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.631882 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.631884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.632297 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.632298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.632435 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.633011 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:54:53.633012 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:53.633043 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:54:53.633065 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:53.633242 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:53.633247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:53.633250 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.633329 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.633431 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260619 08:54:53.634231 2664 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46575
I20260619 08:54:53.634322 2664 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:53.634526 2664 heartbeater.cc:511] Master 127.15.208.62:46575 requested a full tablet report, sending...
I20260619 08:54:53.634873 2345 ts_manager.cc:194] Registered new tserver with Master: 589951a9b2c44526b6485191ea9a04f9 (127.15.208.2:46819)
I20260619 08:54:53.635210 2345 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.2:53941
I20260619 08:54:53.635345 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 2536
I20260619 08:54:53.635468 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:54:53.635689 16192 external_mini_cluster.cc:1428] Setting key b3f23890b785cada6b3d1bed4170c68a
I20260619 08:54:53.636605 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46575
--builtin_ntp_servers=127.15.208.20:40123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:54:53.719031 2667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:53.719262 2667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:53.719286 2667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:53.720593 2667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:53.720638 2667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:53.720666 2667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:54:53.722426 2667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:53.722673 2667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:53.722965 2667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:53.728389 2675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:53.728396 2672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:53.728406 2673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:53.728744 2667 server_base.cc:1034] running on GCE node
I20260619 08:54:53.728969 2667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:53.729195 2667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:53.730336 2667 hybrid_clock.cc:648] HybridClock initialized: now 1781859293730323 us; error 31 us; skew 500 ppm
I20260619 08:54:53.731619 2667 webserver.cc:460] Webserver started at http://127.15.208.3:46167/ using document root <none> and password file <none>
I20260619 08:54:53.731825 2667 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:53.731868 2667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:53.731958 2667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:53.733142 2667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "047eab88169f4b44a72881d460094c98"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "a9bbe25ebe971f5e7268154867f843aa"
server_key_iv: "8e89b738e0f83e2cc0a8ed0ff63c5dd7"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.733443 2667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "047eab88169f4b44a72881d460094c98"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "a9bbe25ebe971f5e7268154867f843aa"
server_key_iv: "8e89b738e0f83e2cc0a8ed0ff63c5dd7"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.734683 2667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:54:53.735548 2681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.735778 2667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:53.735870 2667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "047eab88169f4b44a72881d460094c98"
format_stamp: "Formatted at 2026-06-19 08:54:53 on dist-test-slave-p0h9"
server_key: "a9bbe25ebe971f5e7268154867f843aa"
server_key_iv: "8e89b738e0f83e2cc0a8ed0ff63c5dd7"
server_key_version: "encryptionkey@0"
I20260619 08:54:53.735926 2667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:53.754118 2667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:53.754407 2667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:53.754606 2667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:53.754889 2667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:53.755204 2667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:53.755236 2667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.755259 2667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:53.755271 2667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:53.760926 2667 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:35993
I20260619 08:54:53.761008 2794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:35993 every 8 connection(s)
I20260619 08:54:53.761361 2667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:54:53.766723 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.762134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:43519 (local address 127.15.208.62:46575)
0619 08:54:53.762205 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.762208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.762457 (+ 249us) server_negotiation.cc:407] Connection header received
0619 08:54:53.762941 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.762942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.762952 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.763020 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:53.763744 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.764260 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.765256 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.765475 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.765578 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.766376 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:53.766378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:53.766378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:53.766383 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:53.766400 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:53.766404 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:53.766471 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.766590 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.766634 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:54:53.766921 2797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.762115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46575 (local address 127.15.208.3:43519)
0619 08:54:53.762328 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.762363 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.762844 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.763099 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.763102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.763110 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:53.763639 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.763643 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.764355 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.764356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.765173 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.765176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.765602 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.765603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.765734 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.766235 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:54:53.766235 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:53.766264 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:54:53.766286 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:53.766474 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:53.766480 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:53.766482 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.766577 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.766693 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
I20260619 08:54:53.767373 2795 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46575
I20260619 08:54:53.767450 2795 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:53.767645 2795 heartbeater.cc:511] Master 127.15.208.62:46575 requested a full tablet report, sending...
I20260619 08:54:53.767980 2345 ts_manager.cc:194] Registered new tserver with Master: 047eab88169f4b44a72881d460094c98 (127.15.208.3:35993)
I20260619 08:54:53.768358 2345 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.3:43519
I20260619 08:54:53.771574 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 2667
I20260619 08:54:53.771667 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:54:53.771816 16192 external_mini_cluster.cc:1428] Setting key 8391c87494bd357458423f624dd26980
I20260619 08:54:53.772704 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:54:53.776882 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.774118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39668 (local address 127.15.208.62:46575)
0619 08:54:53.774165 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.774167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.774275 (+ 108us) server_negotiation.cc:407] Connection header received
0619 08:54:53.774324 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.774324 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.774335 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.774379 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:53.774493 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.774941 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.775645 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.775819 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.775868 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.776565 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:53.776567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:53.776567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:53.776570 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:53.776583 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:53.776587 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:53.776664 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.776772 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.776784 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:54:53.778475 2345 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:54:53.780036 2345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:54:53.786283 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.782255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:35993 (local address 127.0.0.1:57210)
0619 08:54:53.782434 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.782491 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.782559 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.782920 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.782923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.782926 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:53.782997 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.783000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.784064 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.784066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.784979 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.784981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.785952 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.785953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.786035 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.786050 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.786134 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.786188 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
I20260619 08:54:53.786386 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.782255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:46819 (local address 127.0.0.1:50982)
0619 08:54:53.782480 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.782491 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.782556 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.782957 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.782958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.782960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:53.782996 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.782998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.784032 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.784033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.785321 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.785324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.786152 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.786153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.786216 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.786219 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.786272 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.786309 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":35,"thread_start_us":93,"threads_started":1}
I20260619 08:54:53.786415 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.782577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50982 (local address 127.15.208.2:46819)
0619 08:54:53.782750 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.782755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.782765 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:53.782806 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.782831 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.782847 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.782968 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:53.783065 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.783933 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.785420 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.786078 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.786143 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.786227 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.786282 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.786327 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260619 08:54:53.786887 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.782360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57210 (local address 127.15.208.3:35993)
0619 08:54:53.782705 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.782710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.782720 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:53.782758 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.782760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.782774 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.782878 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:53.783134 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.783963 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.785169 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.785860 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.785938 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.786042 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.786741 (+ 699us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.786793 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":111,"threads_started":1}
I20260619 08:54:53.787403 2598 tablet_service.cc:1467] Processing CreateTablet for tablet edffe08df7e7485cb7139a1d9ff21130 (DEFAULT_TABLE table=test-table [id=6a15a4ca2ccc4e75adf421d1d823717a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:53.787560 2729 tablet_service.cc:1467] Processing CreateTablet for tablet edffe08df7e7485cb7139a1d9ff21130 (DEFAULT_TABLE table=test-table [id=6a15a4ca2ccc4e75adf421d1d823717a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:53.787760 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.782532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:42459 (local address 127.0.0.1:59944)
0619 08:54:53.782950 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:53.782960 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:53.783003 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:53.784370 (+ 1367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:53.784371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:53.784373 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:53.784407 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.784409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.785653 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.785654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.786748 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:53.786750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.787556 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.787557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:53.787616 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.787619 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:53.787658 (+ 39us) client_negotiation.cc:240] Negotiation successful
0619 08:54:53.787699 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":80,"threads_started":1}
I20260619 08:54:53.787803 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:53.782750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59944 (local address 127.15.208.1:42459)
0619 08:54:53.784149 (+ 1399us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:53.784154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:53.784164 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:53.784205 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:53.784207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:53.784223 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:53.784300 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:53.784483 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.785575 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.786823 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:53.787440 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:53.787559 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:53.787651 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:53.787686 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 08:54:53.787720 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1355,"thread_start_us":105,"threads_started":1}
I20260619 08:54:53.787801 2598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edffe08df7e7485cb7139a1d9ff21130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:53.787849 2729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edffe08df7e7485cb7139a1d9ff21130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:53.791231 2809 tablet_bootstrap.cc:492] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: Bootstrap starting.
I20260619 08:54:53.791972 2809 tablet_bootstrap.cc:654] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:53.792086 2467 tablet_service.cc:1467] Processing CreateTablet for tablet edffe08df7e7485cb7139a1d9ff21130 (DEFAULT_TABLE table=test-table [id=6a15a4ca2ccc4e75adf421d1d823717a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:53.792303 2809 log.cc:824] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:53.792428 2467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edffe08df7e7485cb7139a1d9ff21130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:53.792682 2810 tablet_bootstrap.cc:492] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: Bootstrap starting.
I20260619 08:54:53.793102 2809 tablet_bootstrap.cc:492] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: No bootstrap required, opened a new log
I20260619 08:54:53.793174 2809 ts_tablet_manager.cc:1397] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:54:53.793299 2810 tablet_bootstrap.cc:654] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:53.793633 2810 log.cc:824] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:53.794323 2810 tablet_bootstrap.cc:492] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: No bootstrap required, opened a new log
I20260619 08:54:53.794400 2810 ts_tablet_manager.cc:1397] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 08:54:53.795922 2813 tablet_bootstrap.cc:492] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa: Bootstrap starting.
I20260619 08:54:53.795996 2809 raft_consensus.cc:348] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589951a9b2c44526b6485191ea9a04f9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46819 } } peers { permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 } } peers { permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 } }
I20260619 08:54:53.796188 2809 raft_consensus.cc:374] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:53.796233 2809 raft_consensus.cc:729] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047eab88169f4b44a72881d460094c98, State: Initialized, Role: FOLLOWER
I20260619 08:54:53.796341 2809 consensus_queue.cc:260] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589951a9b2c44526b6485191ea9a04f9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46819 } } peers { permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 } } peers { permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 } }
I20260619 08:54:53.796627 2813 tablet_bootstrap.cc:654] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:53.796625 2809 ts_tablet_manager.cc:1428] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 08:54:53.796643 2795 heartbeater.cc:503] Master 127.15.208.62:46575 was elected leader, sending a full tablet report...
I20260619 08:54:53.796785 2810 raft_consensus.cc:348] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589951a9b2c44526b6485191ea9a04f9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46819 } } peers { permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 } } peers { permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 } }
I20260619 08:54:53.796963 2810 raft_consensus.cc:374] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:53.796979 2813 log.cc:824] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:53.797008 2810 raft_consensus.cc:729] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589951a9b2c44526b6485191ea9a04f9, State: Initialized, Role: FOLLOWER
I20260619 08:54:53.797122 2810 consensus_queue.cc:260] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589951a9b2c44526b6485191ea9a04f9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46819 } } peers { permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 } } peers { permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 } }
I20260619 08:54:53.797402 2664 heartbeater.cc:503] Master 127.15.208.62:46575 was elected leader, sending a full tablet report...
I20260619 08:54:53.797767 2813 tablet_bootstrap.cc:492] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa: No bootstrap required, opened a new log
I20260619 08:54:53.797783 2810 ts_tablet_manager.cc:1428] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 08:54:53.797842 2813 ts_tablet_manager.cc:1397] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 08:54:53.800004 2813 raft_consensus.cc:348] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589951a9b2c44526b6485191ea9a04f9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46819 } } peers { permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 } } peers { permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 } }
I20260619 08:54:53.800122 2813 raft_consensus.cc:374] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:53.800141 2813 raft_consensus.cc:729] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7f22218da8f45adbdc52587d1a05baa, State: Initialized, Role: FOLLOWER
I20260619 08:54:53.800217 2813 consensus_queue.cc:260] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589951a9b2c44526b6485191ea9a04f9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46819 } } peers { permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 } } peers { permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 } }
I20260619 08:54:53.800400 2813 ts_tablet_manager.cc:1428] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:54:53.800482 2533 heartbeater.cc:503] Master 127.15.208.62:46575 was elected leader, sending a full tablet report...
W20260619 08:54:53.878604 2665 tablet.cc:2369] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:54:53.970290 2534 tablet.cc:2369] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:54:54.011945 2796 tablet.cc:2369] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:54:54.030417 2816 raft_consensus.cc:482] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:54.030622 2816 raft_consensus.cc:504] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589951a9b2c44526b6485191ea9a04f9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46819 } } peers { permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 } } peers { permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 } }
I20260619 08:54:54.030997 2816 leader_election.cc:288] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 047eab88169f4b44a72881d460094c98 (127.15.208.3:35993), e7f22218da8f45adbdc52587d1a05baa (127.15.208.1:42459)
I20260619 08:54:54.034833 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:54.031357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:43647 (local address 127.15.208.1:42459)
0619 08:54:54.031406 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:54.031408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:54.031418 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:54.031451 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:54.031452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:54.031463 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:54.031533 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:54.031709 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:54.032485 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:54.033948 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:54.034487 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:54.034565 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:54.034572 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:54.034726 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 08:54:54.034762 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:54:54.034833 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:54.031281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:50343 (local address 127.15.208.3:35993)
0619 08:54:54.031332 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:54.031334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:54.031652 (+ 318us) server_negotiation.cc:407] Connection header received
0619 08:54:54.031705 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:54.031706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:54.031716 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:54.031773 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:54.031897 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:54.032600 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:54.033948 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:54.034487 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:54.034564 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:54.034572 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:54.034726 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 08:54:54.034762 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 08:54:54.034866 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:54.031216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:42459 (local address 127.15.208.2:43647)
0619 08:54:54.031272 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:54.031284 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:54.031336 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:54.031548 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:54.031549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:54.031551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:54.031625 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:54.031627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:54.032569 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:54.032570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:54.033854 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:54.033856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:54.034573 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:54.034575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:54.034646 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:54.034650 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:54:54.034708 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:54:54.034756 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":4}
I20260619 08:54:54.034873 2818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:54.031222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:35993 (local address 127.15.208.2:50343)
0619 08:54:54.031549 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:54.031580 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:54.031638 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:54.031781 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:54.031781 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:54.031783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:54.031828 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:54.031829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:54.032671 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:54.032672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:54.033853 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:54.033856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:54.034589 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:54.034590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:54.034655 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:54.034659 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:54:54.034708 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:54:54.034755 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":129,"thread_start_us":125,"threads_started":1}
I20260619 08:54:54.035271 2749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edffe08df7e7485cb7139a1d9ff21130" candidate_uuid: "589951a9b2c44526b6485191ea9a04f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047eab88169f4b44a72881d460094c98" is_pre_election: true
I20260619 08:54:54.035267 2487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edffe08df7e7485cb7139a1d9ff21130" candidate_uuid: "589951a9b2c44526b6485191ea9a04f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f22218da8f45adbdc52587d1a05baa" is_pre_election: true
I20260619 08:54:54.035423 2487 raft_consensus.cc:2454] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589951a9b2c44526b6485191ea9a04f9 in term 0.
I20260619 08:54:54.035423 2749 raft_consensus.cc:2454] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589951a9b2c44526b6485191ea9a04f9 in term 0.
I20260619 08:54:54.035689 2553 leader_election.cc:302] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047eab88169f4b44a72881d460094c98, 589951a9b2c44526b6485191ea9a04f9; no voters:
I20260619 08:54:54.035862 2816 raft_consensus.cc:2781] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:54:54.035899 2816 raft_consensus.cc:482] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:54:54.035915 2816 raft_consensus.cc:3037] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:54.036620 2816 raft_consensus.cc:504] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589951a9b2c44526b6485191ea9a04f9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46819 } } peers { permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 } } peers { permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 } }
I20260619 08:54:54.036795 2816 leader_election.cc:288] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [CANDIDATE]: Term 1 election: Requested vote from peers 047eab88169f4b44a72881d460094c98 (127.15.208.3:35993), e7f22218da8f45adbdc52587d1a05baa (127.15.208.1:42459)
I20260619 08:54:54.036900 2749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edffe08df7e7485cb7139a1d9ff21130" candidate_uuid: "589951a9b2c44526b6485191ea9a04f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047eab88169f4b44a72881d460094c98"
I20260619 08:54:54.036993 2749 raft_consensus.cc:3037] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:54.037711 2749 raft_consensus.cc:2454] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589951a9b2c44526b6485191ea9a04f9 in term 1.
I20260619 08:54:54.037870 2553 leader_election.cc:302] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047eab88169f4b44a72881d460094c98, 589951a9b2c44526b6485191ea9a04f9; no voters:
I20260619 08:54:54.037919 2487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edffe08df7e7485cb7139a1d9ff21130" candidate_uuid: "589951a9b2c44526b6485191ea9a04f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f22218da8f45adbdc52587d1a05baa"
I20260619 08:54:54.037966 2816 raft_consensus.cc:2781] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:54.038018 2487 raft_consensus.cc:3037] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:54.038154 2816 raft_consensus.cc:686] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 1 LEADER]: Becoming Leader. State: Replica: 589951a9b2c44526b6485191ea9a04f9, State: Running, Role: LEADER
I20260619 08:54:54.038249 2816 consensus_queue.cc:237] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589951a9b2c44526b6485191ea9a04f9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46819 } } peers { permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 } } peers { permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 } }
I20260619 08:54:54.039028 2487 raft_consensus.cc:2454] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589951a9b2c44526b6485191ea9a04f9 in term 1.
I20260619 08:54:54.039705 2344 catalog_manager.cc:5495] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 589951a9b2c44526b6485191ea9a04f9 (127.15.208.2). New cstate: current_term: 1 leader_uuid: "589951a9b2c44526b6485191ea9a04f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589951a9b2c44526b6485191ea9a04f9" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 } health_report { overall_health: UNKNOWN } } }
I20260619 08:54:54.070027 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:54.067815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.15.208.2:46819)
0619 08:54:54.067875 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:54.067877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:54.067887 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:54.067958 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:54.067958 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:54.067977 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:54.068041 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:54:54.068150 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:54.068600 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:54.069341 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:54.069520 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:54.069579 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:54.069684 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:54:54.069766 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:54:54.069833 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:54.069871 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 08:54:54.069879 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:54:54.071825 2749 raft_consensus.cc:1261] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98 [term 1 FOLLOWER]: Refusing update from remote peer 589951a9b2c44526b6485191ea9a04f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:54.071834 2487 raft_consensus.cc:1261] T edffe08df7e7485cb7139a1d9ff21130 P e7f22218da8f45adbdc52587d1a05baa [term 1 FOLLOWER]: Refusing update from remote peer 589951a9b2c44526b6485191ea9a04f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:54.072204 2816 consensus_queue.cc:1035] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047eab88169f4b44a72881d460094c98" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 35993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:54.072357 2819 consensus_queue.cc:1035] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7f22218da8f45adbdc52587d1a05baa" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:54.077539 2345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:54.077641 2345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:54.078365 2345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10da46d3ecdd420fbac697a25ba08ded: Sending DeleteTablet for 3 replicas of tablet edffe08df7e7485cb7139a1d9ff21130
I20260619 08:54:54.078759 2729 tablet_service.cc:1514] Processing DeleteTablet for tablet edffe08df7e7485cb7139a1d9ff21130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:54 UTC) from {username='slave'} at 127.0.0.1:57210
I20260619 08:54:54.078871 2598 tablet_service.cc:1514] Processing DeleteTablet for tablet edffe08df7e7485cb7139a1d9ff21130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:54 UTC) from {username='slave'} at 127.0.0.1:50982
I20260619 08:54:54.079064 2467 tablet_service.cc:1514] Processing DeleteTablet for tablet edffe08df7e7485cb7139a1d9ff21130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:54 UTC) from {username='slave'} at 127.0.0.1:59944
I20260619 08:54:54.079126 2832 tablet_replica.cc:330] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: stopping tablet replica
I20260619 08:54:54.079126 2833 tablet_replica.cc:330] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: stopping tablet replica
I20260619 08:54:54.079227 2832 raft_consensus.cc:2229] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:54.079324 2832 raft_consensus.cc:2258] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:54.079366 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 2405
I20260619 08:54:54.079401 2833 raft_consensus.cc:2229] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:54:54.079563 2833 raft_consensus.cc:2258] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:54.080595 2833 ts_tablet_manager.cc:1905] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:54.081106 2832 ts_tablet_manager.cc:1905] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:54.082389 2833 ts_tablet_manager.cc:1918] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:54.082571 2833 log.cc:1196] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/edffe08df7e7485cb7139a1d9ff21130
I20260619 08:54:54.082739 2832 ts_tablet_manager.cc:1918] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:54.082811 2832 log.cc:1196] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/edffe08df7e7485cb7139a1d9ff21130
I20260619 08:54:54.083027 2833 ts_tablet_manager.cc:1939] T edffe08df7e7485cb7139a1d9ff21130 P 589951a9b2c44526b6485191ea9a04f9: Deleting consensus metadata
I20260619 08:54:54.083036 2832 ts_tablet_manager.cc:1939] T edffe08df7e7485cb7139a1d9ff21130 P 047eab88169f4b44a72881d460094c98: Deleting consensus metadata
I20260619 08:54:54.083442 2331 catalog_manager.cc:4841] TS 047eab88169f4b44a72881d460094c98 (127.15.208.3:35993): tablet edffe08df7e7485cb7139a1d9ff21130 (table test-table [id=6a15a4ca2ccc4e75adf421d1d823717a]) successfully deleted
I20260619 08:54:54.083633 2329 catalog_manager.cc:4841] TS 589951a9b2c44526b6485191ea9a04f9 (127.15.208.2:46819): tablet edffe08df7e7485cb7139a1d9ff21130 (table test-table [id=6a15a4ca2ccc4e75adf421d1d823717a]) successfully deleted
W20260619 08:54:54.085423 2329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:42459 (error 108)
I20260619 08:54:54.085675 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 2536
I20260619 08:54:54.085880 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:54.085757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:42459 (local address 127.0.0.1:59948)
0619 08:54:54.085810 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:54.085844 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:54:54.087597 2329 catalog_manager.cc:4584] TS e7f22218da8f45adbdc52587d1a05baa (127.15.208.1:42459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet edffe08df7e7485cb7139a1d9ff21130: Network error: Client connection negotiation failed: client connection to 127.15.208.1:42459: connect: Connection refused (error 111)
I20260619 08:54:54.089850 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 2667
I20260619 08:54:54.095327 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 2314
2026-06-19T08:54:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (931 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:54:54 dist-test-slave-p0h9 krb5kdc[2838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:54:54 dist-test-slave-p0h9 krb5kdc[2838](info): set up 2 sockets
Jun 19 08:54:54 dist-test-slave-p0h9 krb5kdc[2838](info): commencing operation
krb5kdc: starting...
W20260619 08:54:55.129110 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:54:55 dist-test-slave-p0h9 krb5kdc[2838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:54:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:54:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:54:55.247116 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46233
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:46233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:55.329214 2851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:55.329448 2851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:55.329492 2851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:55.330660 2851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:54:55.330725 2851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:55.330749 2851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:55.330770 2851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:54:55.330797 2851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:54:55.332278 2851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:46233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:46233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:55.332551 2851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:55.332845 2851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:54:55.337852 2851 server_base.cc:1034] running on GCE node
W20260619 08:54:55.337772 2859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:55.337782 2857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:55.337800 2856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:55.338267 2851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:55.338598 2851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:55.339766 2851 hybrid_clock.cc:648] HybridClock initialized: now 1781859295339744 us; error 46 us; skew 500 ppm
Jun 19 08:54:55 dist-test-slave-p0h9 krb5kdc[2838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859295, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:55.342530 2851 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:54:55.342988 2851 webserver.cc:460] Webserver started at http://127.15.208.62:41139/ using document root <none> and password file <none>
I20260619 08:54:55.343245 2851 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:55.343312 2851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:55.343464 2851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:55.344527 2851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "52301f9ab5c747e1b275e25133a756e9"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "819ea91d446dcab24ce8feff240457f4"
server_key_iv: "e71fa5424972b750b345b222e43e3e72"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.344920 2851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "52301f9ab5c747e1b275e25133a756e9"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "819ea91d446dcab24ce8feff240457f4"
server_key_iv: "e71fa5424972b750b345b222e43e3e72"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.346239 2851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:55.347014 2866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:55.347294 2851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:55.347397 2851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "52301f9ab5c747e1b275e25133a756e9"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "819ea91d446dcab24ce8feff240457f4"
server_key_iv: "e71fa5424972b750b345b222e43e3e72"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.347466 2851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:55.357326 2851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:55.359102 2851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:55.359261 2851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:55.363524 2851 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:46233
I20260619 08:54:55.363559 2918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:46233 every 8 connection(s)
I20260619 08:54:55.363914 2851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:54:55.364822 2919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:55.367072 2919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9: Bootstrap starting.
I20260619 08:54:55.367594 2919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:55.367825 2919 log.cc:824] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:55.368479 2919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9: No bootstrap required, opened a new log
I20260619 08:54:55.370108 2919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52301f9ab5c747e1b275e25133a756e9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46233 } }
I20260619 08:54:55.370256 2919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:55.370278 2919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52301f9ab5c747e1b275e25133a756e9, State: Initialized, Role: FOLLOWER
I20260619 08:54:55.370409 2919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52301f9ab5c747e1b275e25133a756e9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46233 } }
I20260619 08:54:55.370487 2919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:54:55.370512 2919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:54:55.370540 2919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:55.371037 2919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52301f9ab5c747e1b275e25133a756e9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46233 } }
I20260619 08:54:55.371119 2919 leader_election.cc:302] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52301f9ab5c747e1b275e25133a756e9; no voters:
I20260619 08:54:55.371249 2919 leader_election.cc:288] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:54:55.371320 2922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:55.371467 2922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [term 1 LEADER]: Becoming Leader. State: Replica: 52301f9ab5c747e1b275e25133a756e9, State: Running, Role: LEADER
I20260619 08:54:55.371503 2919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:54:55.371554 2922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52301f9ab5c747e1b275e25133a756e9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46233 } }
I20260619 08:54:55.371932 2924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52301f9ab5c747e1b275e25133a756e9. Latest consensus state: current_term: 1 leader_uuid: "52301f9ab5c747e1b275e25133a756e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52301f9ab5c747e1b275e25133a756e9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46233 } } }
I20260619 08:54:55.372017 2924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:55.372161 2923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52301f9ab5c747e1b275e25133a756e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52301f9ab5c747e1b275e25133a756e9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 46233 } } }
I20260619 08:54:55.372231 2923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:55.372367 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 2851
I20260619 08:54:55.372485 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:54:55.372690 16192 external_mini_cluster.cc:1428] Setting key abb483376e47e09866c2d4d50e2e7dde
I20260619 08:54:55.373109 2930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:54:55.373677 2930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:54:55.375195 2930 catalog_manager.cc:1347] Generated new cluster ID: 2716d827d6ed45ff81735480ac505b8c
I20260619 08:54:55.375247 2930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:54:55 dist-test-slave-p0h9 krb5kdc[2838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:55.391616 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.373208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40798 (local address 127.15.208.62:46233)
0619 08:54:55.373386 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:55.373399 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:55.373603 (+ 204us) server_negotiation.cc:407] Connection header received
0619 08:54:55.373733 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:55.373740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:55.373805 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:55.373945 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:55.381645 (+ 7700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.383051 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.384163 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.384809 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.384900 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.387767 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:55.387772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:55.387772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:55.387785 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:55.389826 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:55.390215 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:55.390218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:55.390219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:55.390343 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:55.390530 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:55.390530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:55.390531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:55.390865 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:55.391043 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:55.391112 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 08:54:55.391363 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260619 08:54:55.397851 2930 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:54:55.398398 2930 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:54:55.409313 2930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9: Generated new TSK 0
I20260619 08:54:55.409520 2930 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
Entry for principal kudu/127.15.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab.
I20260619 08:54:55.443053 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46233
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:55.525172 2945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:55.525383 2945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:55.525408 2945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:55.527102 2945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:55.527159 2945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:55.527210 2945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:54:55.528716 2945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.1.keytab
--principal=kudu/127.15.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:55.529009 2945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:55.529247 2945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:55.534264 2953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:55.534289 2945 server_base.cc:1034] running on GCE node
W20260619 08:54:55.534430 2950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:55.534265 2951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:55.534724 2945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:55.535009 2945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:55.536165 2945 hybrid_clock.cc:648] HybridClock initialized: now 1781859295536155 us; error 40 us; skew 500 ppm
Jun 19 08:54:55 dist-test-slave-p0h9 krb5kdc[2838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859295, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:55.538667 2945 init.cc:377] Logged in from keytab as kudu/127.15.208.1@KRBTEST.COM (short username kudu)
I20260619 08:54:55.539103 2945 webserver.cc:460] Webserver started at http://127.15.208.1:32955/ using document root <none> and password file <none>
I20260619 08:54:55.539321 2945 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:55.539386 2945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:55.539527 2945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:55.540575 2945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "8b0b909a4cb14265a0b69d24188a1d9f"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "83085448415b587b779b2c66a98684c4"
server_key_iv: "6f9b449eb570c2648c09d4687894e3dd"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.540957 2945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "8b0b909a4cb14265a0b69d24188a1d9f"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "83085448415b587b779b2c66a98684c4"
server_key_iv: "6f9b449eb570c2648c09d4687894e3dd"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.542285 2945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:55.543102 2960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:55.543308 2945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:55.543411 2945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "8b0b909a4cb14265a0b69d24188a1d9f"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "83085448415b587b779b2c66a98684c4"
server_key_iv: "6f9b449eb570c2648c09d4687894e3dd"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.543493 2945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:55.570377 2945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:55.572428 2945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:55.572604 2945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:55.572942 2945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:55.573271 2945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:55.573323 2945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:55.573359 2945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:55.573385 2945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:55.579715 2945 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:43223
I20260619 08:54:55.580096 3073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:43223 every 8 connection(s)
I20260619 08:54:55.580163 2945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:54:55 dist-test-slave-p0h9 krb5kdc[2838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859295, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.1@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:55.588557 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 2945
I20260619 08:54:55.588673 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:54:55.588891 16192 external_mini_cluster.cc:1428] Setting key a9227e626b7172515db1064c83acaeee
I20260619 08:54:55.589257 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.581002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:54407 (local address 127.15.208.62:46233)
0619 08:54:55.581074 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:55.581089 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:55.581260 (+ 171us) server_negotiation.cc:407] Connection header received
0619 08:54:55.581821 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:55.581823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:55.581836 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:55.581897 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:55.582797 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.583312 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.584260 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.584452 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.584507 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.586629 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:55.586630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:55.586631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:55.586638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:55.587806 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:55.588173 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:55.588174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:55.588174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:55.588223 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:55.588488 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:55.588489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:55.588489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:55.588633 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:55.588698 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:55.589069 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 08:54:55.589168 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:54:55.589428 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.580948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46233 (local address 127.15.208.1:54407)
0619 08:54:55.581126 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:55.581170 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:55.581722 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:55.581949 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:55.581952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:55.582520 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:55.582719 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.582724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.583389 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.583390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.584186 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.584188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.584526 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.584526 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.584634 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.585163 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:55.585171 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:55.586531 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:55.587875 (+ 1344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:55.587881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:55.587883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:55.588110 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:55.588277 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:55.588277 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:55.588277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:55.588420 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:55.588731 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:55.588757 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:55.588897 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 08:54:55.589019 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 08:54:55.589189 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
I20260619 08:54:55.591135 3074 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46233
I20260619 08:54:55.591253 3074 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:55.591430 3074 heartbeater.cc:511] Master 127.15.208.62:46233 requested a full tablet report, sending...
I20260619 08:54:55.591974 2883 ts_manager.cc:194] Registered new tserver with Master: 8b0b909a4cb14265a0b69d24188a1d9f (127.15.208.1:43223)
I20260619 08:54:55.592594 2883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.1@KRBTEST.COM'} at 127.15.208.1:54407
WARNING: no policy specified for kudu/127.15.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
Entry for principal kudu/127.15.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab.
I20260619 08:54:55.614814 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46233
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:55.695297 3079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:55.695541 3079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:55.695592 3079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:55.696787 3079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:55.696847 3079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:55.696890 3079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:54:55.698372 3079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.2.keytab
--principal=kudu/127.15.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:55.698654 3079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:55.698892 3079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:55.703819 3087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:55.703824 3085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:55.703824 3084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:55.704177 3079 server_base.cc:1034] running on GCE node
I20260619 08:54:55.704393 3079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:55.704634 3079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:55.705797 3079 hybrid_clock.cc:648] HybridClock initialized: now 1781859295705779 us; error 30 us; skew 500 ppm
Jun 19 08:54:55 dist-test-slave-p0h9 krb5kdc[2838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859295, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:55.708325 3079 init.cc:377] Logged in from keytab as kudu/127.15.208.2@KRBTEST.COM (short username kudu)
I20260619 08:54:55.708866 3079 webserver.cc:460] Webserver started at http://127.15.208.2:34477/ using document root <none> and password file <none>
I20260619 08:54:55.709095 3079 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:55.709158 3079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:55.709292 3079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:55.710306 3079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "156a8499f1b14064bc5c5164d47bca29"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "84b3773f9ea93ce65d6bef1efe921a67"
server_key_iv: "ec9cff4497caac695a50a0c702d18f58"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.710672 3079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "156a8499f1b14064bc5c5164d47bca29"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "84b3773f9ea93ce65d6bef1efe921a67"
server_key_iv: "ec9cff4497caac695a50a0c702d18f58"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.711971 3079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:54:55.712709 3094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:55.712925 3079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:55.713027 3079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "156a8499f1b14064bc5c5164d47bca29"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "84b3773f9ea93ce65d6bef1efe921a67"
server_key_iv: "ec9cff4497caac695a50a0c702d18f58"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.713107 3079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:55.721004 3079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:55.723217 3079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:55.723371 3079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:55.723595 3079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:55.723894 3079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:55.723944 3079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:55.723984 3079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:55.724012 3079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:55.730008 3079 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:45867
I20260619 08:54:55.730049 3207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:45867 every 8 connection(s)
I20260619 08:54:55.730365 3079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:54:55 dist-test-slave-p0h9 krb5kdc[2838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859295, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.2@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:55.739601 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 3079
I20260619 08:54:55.739733 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:54:55.739916 16192 external_mini_cluster.cc:1428] Setting key ae995d15b48316cc7741c534d4b8304d
I20260619 08:54:55.740213 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.731099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:36247 (local address 127.15.208.62:46233)
0619 08:54:55.731140 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:55.731142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:55.731422 (+ 280us) server_negotiation.cc:407] Connection header received
0619 08:54:55.732151 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:55.732152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:55.732165 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:55.732223 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:55.733284 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.733788 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.734887 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.735075 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.735154 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.737389 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:55.737390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:55.737391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:55.737398 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:55.738811 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:55.739235 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:55.739236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:55.739236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:55.739293 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:55.739537 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:55.739538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:55.739539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:55.739656 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:55.739724 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:55.740027 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 08:54:55.740115 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:54:55.740406 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.731082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46233 (local address 127.15.208.2:36247)
0619 08:54:55.731277 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:55.731324 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:55.732044 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:55.732288 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:55.732292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:55.732971 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:55.733175 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.733179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.733882 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.733883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.734791 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.734794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.735163 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.735163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.735289 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.735829 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:55.735838 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:55.737295 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:55.738913 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:55.738920 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:55.738922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:55.739148 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:55.739361 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:55.739362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:55.739362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:55.739469 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:55.739727 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:55.739734 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:55.739896 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 08:54:55.740007 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 08:54:55.740182 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260619 08:54:55.740862 3208 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46233
I20260619 08:54:55.740978 3208 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:55.741175 3208 heartbeater.cc:511] Master 127.15.208.62:46233 requested a full tablet report, sending...
I20260619 08:54:55.741585 2883 ts_manager.cc:194] Registered new tserver with Master: 156a8499f1b14064bc5c5164d47bca29 (127.15.208.2:45867)
I20260619 08:54:55.741946 2883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.2@KRBTEST.COM'} at 127.15.208.2:36247
WARNING: no policy specified for kudu/127.15.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
Entry for principal kudu/127.15.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab.
I20260619 08:54:55.764837 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46233
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:55.845566 3213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:55.845786 3213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:55.845826 3213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:55.847069 3213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:55.847128 3213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:55.847173 3213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:54:55.848599 3213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.3.keytab
--principal=kudu/127.15.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:46233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:55.848873 3213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:55.849090 3213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:55.854205 3219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:55.854207 3218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:55.854239 3213 server_base.cc:1034] running on GCE node
W20260619 08:54:55.854221 3221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:55.854738 3213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:55.854974 3213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:55.856137 3213 hybrid_clock.cc:648] HybridClock initialized: now 1781859295856126 us; error 42 us; skew 500 ppm
Jun 19 08:54:55 dist-test-slave-p0h9 krb5kdc[2838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859295, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:55.858708 3213 init.cc:377] Logged in from keytab as kudu/127.15.208.3@KRBTEST.COM (short username kudu)
I20260619 08:54:55.859179 3213 webserver.cc:460] Webserver started at http://127.15.208.3:42559/ using document root <none> and password file <none>
I20260619 08:54:55.859404 3213 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:55.859484 3213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:55.859632 3213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:55.860648 3213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "02c406826b51427f94ed8afbcdbe168d"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "637e8240472da52bfff81341e2590b4f"
server_key_iv: "5b88a20baf22aec40ad61e4062ea85ee"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.861112 3213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "02c406826b51427f94ed8afbcdbe168d"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "637e8240472da52bfff81341e2590b4f"
server_key_iv: "5b88a20baf22aec40ad61e4062ea85ee"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.862432 3213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:55.863226 3228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:55.863410 3213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:55.863498 3213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "02c406826b51427f94ed8afbcdbe168d"
format_stamp: "Formatted at 2026-06-19 08:54:55 on dist-test-slave-p0h9"
server_key: "637e8240472da52bfff81341e2590b4f"
server_key_iv: "5b88a20baf22aec40ad61e4062ea85ee"
server_key_version: "encryptionkey@0"
I20260619 08:54:55.863576 3213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:55.881762 3213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:55.884153 3213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:55.884315 3213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:55.884581 3213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:55.884980 3213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:55.885037 3213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:55.885079 3213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:55.885114 3213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:55.891184 3213 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:39007
I20260619 08:54:55.891227 3341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:39007 every 8 connection(s)
I20260619 08:54:55.891865 3213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:54:55 dist-test-slave-p0h9 krb5kdc[2838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859295, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.3@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:54:55.899710 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 3213
I20260619 08:54:55.899825 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:54:55.900008 16192 external_mini_cluster.cc:1428] Setting key 4954a86a6d078f01d5d2396bc8732165
I20260619 08:54:55.901603 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.892615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:55695 (local address 127.15.208.62:46233)
0619 08:54:55.892676 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:55.892678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:55.892925 (+ 247us) server_negotiation.cc:407] Connection header received
0619 08:54:55.893470 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:55.893471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:55.893484 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:55.893584 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:55.894533 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.895031 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.896130 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.896312 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.896404 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.898622 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:55.898624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:55.898624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:55.898632 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:55.900060 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:55.900511 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:55.900512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:55.900512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:55.900589 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:55.900897 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:55.900899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:55.900899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:55.901051 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:55.901134 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:55.901415 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 08:54:55.901504 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:54:55.901844 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.892605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:46233 (local address 127.15.208.3:55695)
0619 08:54:55.892802 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:55.892847 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:55.893358 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:55.893657 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:55.893662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:55.894228 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:55.894417 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.894421 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.895186 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.895187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.896028 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.896030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.896412 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.896413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.896542 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.897056 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:54:55.897064 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:55.898455 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:55.900182 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:55.900194 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:55.900195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:55.900423 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:55.900655 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:54:55.900656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:54:55.900656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:54:55.900812 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:54:55.901123 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:55.901130 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:55.901270 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 08:54:55.901374 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 08:54:55.901588 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260619 08:54:55.902319 3342 heartbeater.cc:347] Connected to a master server at 127.15.208.62:46233
I20260619 08:54:55.902421 3342 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:55.902616 3342 heartbeater.cc:511] Master 127.15.208.62:46233 requested a full tablet report, sending...
I20260619 08:54:55.902983 2883 ts_manager.cc:194] Registered new tserver with Master: 02c406826b51427f94ed8afbcdbe168d (127.15.208.3:39007)
I20260619 08:54:55.903335 2883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.208.3@KRBTEST.COM'} at 127.15.208.3:55695
I20260619 08:54:55.911759 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:54:55.919438 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.913161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40814 (local address 127.15.208.62:46233)
0619 08:54:55.913202 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:55.913203 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:55.913320 (+ 117us) server_negotiation.cc:407] Connection header received
0619 08:54:55.913377 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:55.913378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:55.913386 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:55.913437 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:55.914081 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.914599 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.915457 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.915731 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.915791 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.917109 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:55.917110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:55.917111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:55.917116 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:55.918484 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:55.918804 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:55.918805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:55.918806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:55.918853 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:55.919013 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:55.919013 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:55.919013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:55.919143 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:55.919213 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:55.919310 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 08:54:55.919379 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:54:55.921196 2883 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:54:55.922721 2883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:54:55.929351 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.925140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:39007 (local address 127.0.0.1:48540)
0619 08:54:55.925358 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:55.925367 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:55.925419 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:55.925745 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:55.925746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:55.925748 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:55.925812 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.925814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.926796 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.926797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.928166 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.928169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.929035 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.929036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.929121 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.929136 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 08:54:55.929217 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:54:55.929272 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":10,"thread_start_us":76,"threads_started":1}
I20260619 08:54:55.929374 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.925347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48540 (local address 127.15.208.3:39007)
0619 08:54:55.925581 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:55.925586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:55.925597 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:54:55.925638 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:55.925640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:55.925658 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:55.925750 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:55.925879 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.926691 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.928245 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.928951 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.929030 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.929088 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:55.929243 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 08:54:55.929297 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":96,"threads_started":1}
I20260619 08:54:55.929548 3353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.924934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43223 (local address 127.0.0.1:34556)
0619 08:54:55.925111 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:55.925154 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:55.925238 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:55.925701 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:55.925704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:55.925707 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:55.925777 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.925779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.927222 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.927223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.928372 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.928374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.929326 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.929327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.929393 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.929398 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:54:55.929430 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 08:54:55.929477 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":62,"threads_started":1}
I20260619 08:54:55.929797 3355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.925025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34556 (local address 127.15.208.1:43223)
0619 08:54:55.925466 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:55.925471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:55.925482 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:54:55.925558 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:55.925561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:55.925577 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:55.925967 (+ 390us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:55.926039 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.927112 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.928461 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.929227 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.929608 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.929660 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:55.929680 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 08:54:55.929709 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":95,"threads_started":1}
I20260619 08:54:55.930617 3356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.924994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:45867 (local address 127.0.0.1:53780)
0619 08:54:55.925271 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:55.925281 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:55.925329 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:55.925746 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:55.925747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:55.925748 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:55.925804 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.925805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.928462 (+ 2657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.928463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.929435 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:55.929438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.930442 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.930443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:55.930491 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.930494 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:55.930542 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:54:55.930570 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":14,"thread_start_us":111,"threads_started":1}
I20260619 08:54:55.930685 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:55.925225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.15.208.2:45867)
0619 08:54:55.925589 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:55.925593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:55.925603 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:55.925639 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:55.925640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:55.925655 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:55.925876 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:55.925952 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.927100 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.929548 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:55.930361 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:55.930444 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:55.930501 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:55.930557 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 08:54:55.930606 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":96,"threads_started":1}
I20260619 08:54:55.930580 3276 tablet_service.cc:1467] Processing CreateTablet for tablet 74167893e9bd4535a4890e33d7e905fa (DEFAULT_TABLE table=test-table [id=d6a162da6aad4c38bc4fa63652bec3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:55.930982 3276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74167893e9bd4535a4890e33d7e905fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:55.931329 3142 tablet_service.cc:1467] Processing CreateTablet for tablet 74167893e9bd4535a4890e33d7e905fa (DEFAULT_TABLE table=test-table [id=d6a162da6aad4c38bc4fa63652bec3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:55.931658 3142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74167893e9bd4535a4890e33d7e905fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:55.933184 3008 tablet_service.cc:1467] Processing CreateTablet for tablet 74167893e9bd4535a4890e33d7e905fa (DEFAULT_TABLE table=test-table [id=d6a162da6aad4c38bc4fa63652bec3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:55.933408 3008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74167893e9bd4535a4890e33d7e905fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:55.935523 3360 tablet_bootstrap.cc:492] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: Bootstrap starting.
I20260619 08:54:55.935514 3359 tablet_bootstrap.cc:492] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: Bootstrap starting.
I20260619 08:54:55.936250 3359 tablet_bootstrap.cc:654] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:55.936250 3360 tablet_bootstrap.cc:654] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:55.936875 3361 tablet_bootstrap.cc:492] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f: Bootstrap starting.
I20260619 08:54:55.936937 3359 log.cc:824] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:55.936951 3360 log.cc:824] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:55.937536 3361 tablet_bootstrap.cc:654] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:55.937763 3360 tablet_bootstrap.cc:492] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: No bootstrap required, opened a new log
I20260619 08:54:55.937848 3360 ts_tablet_manager.cc:1397] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:54:55.938084 3359 tablet_bootstrap.cc:492] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: No bootstrap required, opened a new log
I20260619 08:54:55.938105 3361 log.cc:824] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:55.938160 3359 ts_tablet_manager.cc:1397] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:54:55.938763 3361 tablet_bootstrap.cc:492] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f: No bootstrap required, opened a new log
I20260619 08:54:55.938838 3361 ts_tablet_manager.cc:1397] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:54:55.940034 3360 raft_consensus.cc:348] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43223 } } peers { permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 } } peers { permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 } }
I20260619 08:54:55.940233 3360 raft_consensus.cc:374] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:55.940272 3360 raft_consensus.cc:729] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 156a8499f1b14064bc5c5164d47bca29, State: Initialized, Role: FOLLOWER
I20260619 08:54:55.940209 3361 raft_consensus.cc:348] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43223 } } peers { permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 } } peers { permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 } }
I20260619 08:54:55.940445 3360 consensus_queue.cc:260] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43223 } } peers { permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 } } peers { permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 } }
I20260619 08:54:55.940443 3359 raft_consensus.cc:348] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43223 } } peers { permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 } } peers { permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 } }
I20260619 08:54:55.940550 3361 raft_consensus.cc:374] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:55.940613 3361 raft_consensus.cc:729] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0b909a4cb14265a0b69d24188a1d9f, State: Initialized, Role: FOLLOWER
I20260619 08:54:55.940611 3359 raft_consensus.cc:374] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:55.940658 3359 raft_consensus.cc:729] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c406826b51427f94ed8afbcdbe168d, State: Initialized, Role: FOLLOWER
I20260619 08:54:55.940747 3361 consensus_queue.cc:260] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43223 } } peers { permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 } } peers { permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 } }
I20260619 08:54:55.940807 3360 ts_tablet_manager.cc:1428] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:54:55.940845 3359 consensus_queue.cc:260] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43223 } } peers { permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 } } peers { permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 } }
I20260619 08:54:55.940945 3208 heartbeater.cc:503] Master 127.15.208.62:46233 was elected leader, sending a full tablet report...
I20260619 08:54:55.941046 3361 ts_tablet_manager.cc:1428] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:54:55.941180 3074 heartbeater.cc:503] Master 127.15.208.62:46233 was elected leader, sending a full tablet report...
I20260619 08:54:55.941205 3359 ts_tablet_manager.cc:1428] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:54:55.941325 3342 heartbeater.cc:503] Master 127.15.208.62:46233 was elected leader, sending a full tablet report...
W20260619 08:54:55.980966 3209 tablet.cc:2369] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:54:56.036329 3366 raft_consensus.cc:482] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:56.036495 3366 raft_consensus.cc:504] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43223 } } peers { permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 } } peers { permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 } }
I20260619 08:54:56.036890 3366 leader_election.cc:288] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c406826b51427f94ed8afbcdbe168d (127.15.208.3:39007), 156a8499f1b14064bc5c5164d47bca29 (127.15.208.2:45867)
I20260619 08:54:56.039934 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.037253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:47965 (local address 127.15.208.3:39007)
0619 08:54:56.037311 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.037313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.037323 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:56.037353 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.037354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.037362 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.037415 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:56.037564 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.038023 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.038999 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.039592 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.039657 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.039666 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.039833 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.039868 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 08:54:56.039944 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.037084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:39007 (local address 127.15.208.1:47965)
0619 08:54:56.037169 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.037182 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:56.037249 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:56.037418 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:56.037419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:56.037420 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:56.037501 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.037503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.038108 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.038109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.038882 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.038884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.039665 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.039666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.039755 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.039758 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:56.039815 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 08:54:56.039859 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":26}
I20260619 08:54:56.040004 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.037194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:52417 (local address 127.15.208.2:45867)
0619 08:54:56.037329 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.037331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.037434 (+ 103us) server_negotiation.cc:407] Connection header received
0619 08:54:56.037489 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.037490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.037502 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.037555 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:56.037712 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.038153 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.039103 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.039653 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.039731 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.039739 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.039896 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.039933 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260619 08:54:56.040045 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.037084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:45867 (local address 127.15.208.1:52417)
0619 08:54:56.037355 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.037365 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:56.037428 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:56.037559 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:56.037560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:56.037561 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:56.037634 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.037635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.038229 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.038230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.038989 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.038991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.039731 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.039732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.039814 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.039819 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:54:56.039889 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:54:56.039926 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":95,"thread_start_us":112,"threads_started":1}
I20260619 08:54:56.040292 3296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74167893e9bd4535a4890e33d7e905fa" candidate_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c406826b51427f94ed8afbcdbe168d" is_pre_election: true
I20260619 08:54:56.040462 3296 raft_consensus.cc:2454] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b0b909a4cb14265a0b69d24188a1d9f in term 0.
I20260619 08:54:56.040807 2963 leader_election.cc:302] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c406826b51427f94ed8afbcdbe168d, 8b0b909a4cb14265a0b69d24188a1d9f; no voters:
I20260619 08:54:56.040845 3162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74167893e9bd4535a4890e33d7e905fa" candidate_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156a8499f1b14064bc5c5164d47bca29" is_pre_election: true
I20260619 08:54:56.040980 3162 raft_consensus.cc:2454] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b0b909a4cb14265a0b69d24188a1d9f in term 0.
I20260619 08:54:56.041004 3366 raft_consensus.cc:2781] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:54:56.041106 3366 raft_consensus.cc:482] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:54:56.041141 3366 raft_consensus.cc:3037] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:56.041870 3366 raft_consensus.cc:504] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43223 } } peers { permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 } } peers { permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 } }
I20260619 08:54:56.042109 3366 leader_election.cc:288] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [CANDIDATE]: Term 1 election: Requested vote from peers 02c406826b51427f94ed8afbcdbe168d (127.15.208.3:39007), 156a8499f1b14064bc5c5164d47bca29 (127.15.208.2:45867)
I20260619 08:54:56.042181 3296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74167893e9bd4535a4890e33d7e905fa" candidate_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c406826b51427f94ed8afbcdbe168d"
I20260619 08:54:56.042244 3296 raft_consensus.cc:3037] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:56.042220 3162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74167893e9bd4535a4890e33d7e905fa" candidate_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156a8499f1b14064bc5c5164d47bca29"
I20260619 08:54:56.042291 3162 raft_consensus.cc:3037] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:56.042956 3162 raft_consensus.cc:2454] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b0b909a4cb14265a0b69d24188a1d9f in term 1.
I20260619 08:54:56.043124 2964 leader_election.cc:302] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 156a8499f1b14064bc5c5164d47bca29, 8b0b909a4cb14265a0b69d24188a1d9f; no voters:
I20260619 08:54:56.043228 3366 raft_consensus.cc:2781] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:56.043372 3366 raft_consensus.cc:686] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 1 LEADER]: Becoming Leader. State: Replica: 8b0b909a4cb14265a0b69d24188a1d9f, State: Running, Role: LEADER
I20260619 08:54:56.043391 3296 raft_consensus.cc:2454] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b0b909a4cb14265a0b69d24188a1d9f in term 1.
I20260619 08:54:56.043483 3366 consensus_queue.cc:237] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43223 } } peers { permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 } } peers { permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 } }
I20260619 08:54:56.044381 2883 catalog_manager.cc:5495] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b0b909a4cb14265a0b69d24188a1d9f (127.15.208.1). New cstate: current_term: 1 leader_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0b909a4cb14265a0b69d24188a1d9f" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 43223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 } health_report { overall_health: UNKNOWN } } }
I20260619 08:54:56.072348 3355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.070186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34560 (local address 127.15.208.1:43223)
0619 08:54:56.070234 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.070236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.070243 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:54:56.070277 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.070277 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.070295 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.070355 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:54:56.070449 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.070894 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.071709 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.071891 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.071945 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.072078 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:54:56.072173 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:54:56.072228 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.072258 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.072272 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:54:56.074249 3162 raft_consensus.cc:1261] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29 [term 1 FOLLOWER]: Refusing update from remote peer 8b0b909a4cb14265a0b69d24188a1d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:56.074252 3296 raft_consensus.cc:1261] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d [term 1 FOLLOWER]: Refusing update from remote peer 8b0b909a4cb14265a0b69d24188a1d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:56.074638 3369 consensus_queue.cc:1035] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c406826b51427f94ed8afbcdbe168d" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:56.074776 3366 consensus_queue.cc:1035] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "156a8499f1b14064bc5c5164d47bca29" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 45867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:56.078168 3373 mvcc.cc:205] Tried to move back new op lower bound from 7298495676717490176 to 7298495676594745344. Current Snapshot: MvccSnapshot[applied={T|T < 7298495676717490176}]
I20260619 08:54:56.079315 2883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:56.079412 2883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:56.080113 2883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52301f9ab5c747e1b275e25133a756e9: Sending DeleteTablet for 3 replicas of tablet 74167893e9bd4535a4890e33d7e905fa
I20260619 08:54:56.080428 3008 tablet_service.cc:1514] Processing DeleteTablet for tablet 74167893e9bd4535a4890e33d7e905fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:56 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:34556
I20260619 08:54:56.080482 3142 tablet_service.cc:1514] Processing DeleteTablet for tablet 74167893e9bd4535a4890e33d7e905fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:56 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:53780
I20260619 08:54:56.080482 3276 tablet_service.cc:1514] Processing DeleteTablet for tablet 74167893e9bd4535a4890e33d7e905fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:56 UTC) from {username='kudu', principal='kudu/127.15.208.62@KRBTEST.COM'} at 127.0.0.1:48540
I20260619 08:54:56.080634 3381 tablet_replica.cc:330] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f: stopping tablet replica
I20260619 08:54:56.080716 3381 raft_consensus.cc:2229] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:54:56.080857 3381 raft_consensus.cc:2258] T 74167893e9bd4535a4890e33d7e905fa P 8b0b909a4cb14265a0b69d24188a1d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:56.080947 3382 tablet_replica.cc:330] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: stopping tablet replica
I20260619 08:54:56.081054 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 2945
I20260619 08:54:56.081086 3382 raft_consensus.cc:2229] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:56.082938 3382 raft_consensus.cc:2258] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:56.083388 3382 ts_tablet_manager.cc:1905] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:56.084867 3383 tablet_replica.cc:330] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: stopping tablet replica
I20260619 08:54:56.085000 3383 raft_consensus.cc:2229] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:56.085116 3383 raft_consensus.cc:2258] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:56.085374 3382 ts_tablet_manager.cc:1918] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:56.085450 3382 log.cc:1196] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/74167893e9bd4535a4890e33d7e905fa
I20260619 08:54:56.085548 3383 ts_tablet_manager.cc:1905] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:56.085721 3382 ts_tablet_manager.cc:1939] T 74167893e9bd4535a4890e33d7e905fa P 02c406826b51427f94ed8afbcdbe168d: Deleting consensus metadata
I20260619 08:54:56.086105 2869 catalog_manager.cc:4841] TS 02c406826b51427f94ed8afbcdbe168d (127.15.208.3:39007): tablet 74167893e9bd4535a4890e33d7e905fa (table test-table [id=d6a162da6aad4c38bc4fa63652bec3dc]) successfully deleted
I20260619 08:54:56.086833 3383 ts_tablet_manager.cc:1918] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:56.086901 3383 log.cc:1196] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/74167893e9bd4535a4890e33d7e905fa
W20260619 08:54:56.086967 2869 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:43223 (error 108)
I20260619 08:54:56.087198 3383 ts_tablet_manager.cc:1939] T 74167893e9bd4535a4890e33d7e905fa P 156a8499f1b14064bc5c5164d47bca29: Deleting consensus metadata
I20260619 08:54:56.087554 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.087404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:43223 (local address 127.0.0.1:34570)
0619 08:54:56.087459 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.087519 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:43223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:54:56.087565 2870 catalog_manager.cc:4841] TS 156a8499f1b14064bc5c5164d47bca29 (127.15.208.2:45867): tablet 74167893e9bd4535a4890e33d7e905fa (table test-table [id=d6a162da6aad4c38bc4fa63652bec3dc]) successfully deleted
I20260619 08:54:56.087644 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 3079
W20260619 08:54:56.087677 2869 catalog_manager.cc:4584] TS 8b0b909a4cb14265a0b69d24188a1d9f (127.15.208.1:43223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74167893e9bd4535a4890e33d7e905fa: Network error: Client connection negotiation failed: client connection to 127.15.208.1:43223: connect: Connection refused (error 111)
I20260619 08:54:56.092713 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 3213
I20260619 08:54:56.097342 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 2851
2026-06-19T08:54:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2002 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T08:54:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:54:56Z Disabled control of system clock
I20260619 08:54:56.124699 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:34915
--webserver_interface=127.15.208.62
--webserver_port=0
--builtin_ntp_servers=127.15.208.20:37005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:34915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:54:56.204841 3388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:56.205075 3388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:56.205097 3388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:56.206283 3388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:54:56.206329 3388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:56.206342 3388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:56.206355 3388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:54:56.206398 3388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:54:56.207741 3388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:34915
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:34915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:56.208004 3388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:56.208314 3388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:56.213160 3393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:56.213243 3388 server_base.cc:1034] running on GCE node
W20260619 08:54:56.213176 3394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:56.213335 3396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:56.213613 3388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:56.213903 3388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:56.215066 3388 hybrid_clock.cc:648] HybridClock initialized: now 1781859296215048 us; error 38 us; skew 500 ppm
I20260619 08:54:56.216311 3388 webserver.cc:460] Webserver started at http://127.15.208.62:35859/ using document root <none> and password file <none>
I20260619 08:54:56.216562 3388 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:56.216631 3388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:56.216832 3388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:56.217957 3388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "9af296bd358d4219bd49556b50ad970a"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "e79eff6b68c3d47d0565632077e95006"
server_key_iv: "ee29dfad95694a0f4e3250c3b59ca92e"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.218323 3388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "9af296bd358d4219bd49556b50ad970a"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "e79eff6b68c3d47d0565632077e95006"
server_key_iv: "ee29dfad95694a0f4e3250c3b59ca92e"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.219698 3388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:54:56.220420 3402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:56.220584 3388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:54:56.220702 3388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "9af296bd358d4219bd49556b50ad970a"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "e79eff6b68c3d47d0565632077e95006"
server_key_iv: "ee29dfad95694a0f4e3250c3b59ca92e"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.220834 3388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:56.232443 3388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:56.232813 3388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:56.232967 3388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:56.237192 3388 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:34915
I20260619 08:54:56.237242 3454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:34915 every 8 connection(s)
I20260619 08:54:56.237599 3388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:54:56.238498 3455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:56.239012 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 3388
I20260619 08:54:56.239149 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:54:56.239329 16192 external_mini_cluster.cc:1428] Setting key cdb4d54142e9fe572f4f490a5dc37a2c
I20260619 08:54:56.241283 3455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a: Bootstrap starting.
I20260619 08:54:56.241915 3455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:56.242225 3455 log.cc:824] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:56.242959 3455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a: No bootstrap required, opened a new log
I20260619 08:54:56.244905 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.239854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45790 (local address 127.15.208.62:34915)
0619 08:54:56.240041 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.240048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.240080 (+ 32us) server_negotiation.cc:407] Connection header received
0619 08:54:56.240483 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.240489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.240550 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.240660 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:56.240924 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.241800 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.242859 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.243314 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.243446 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.244204 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:56.244208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:56.244209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:56.244366 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:56.244398 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:56.244408 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:56.244472 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.244621 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.244705 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260619 08:54:56.245204 3455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af296bd358d4219bd49556b50ad970a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34915 } }
I20260619 08:54:56.245337 3455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:56.245360 3455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9af296bd358d4219bd49556b50ad970a, State: Initialized, Role: FOLLOWER
I20260619 08:54:56.245510 3455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af296bd358d4219bd49556b50ad970a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34915 } }
I20260619 08:54:56.245606 3455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:54:56.245640 3455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:54:56.245690 3455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:56.246210 3455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af296bd358d4219bd49556b50ad970a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34915 } }
I20260619 08:54:56.246315 3455 leader_election.cc:302] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9af296bd358d4219bd49556b50ad970a; no voters:
I20260619 08:54:56.246491 3455 leader_election.cc:288] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:54:56.246524 3460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:56.246680 3460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [term 1 LEADER]: Becoming Leader. State: Replica: 9af296bd358d4219bd49556b50ad970a, State: Running, Role: LEADER
I20260619 08:54:56.246770 3455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:54:56.246768 3460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af296bd358d4219bd49556b50ad970a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34915 } }
I20260619 08:54:56.247133 3462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9af296bd358d4219bd49556b50ad970a. Latest consensus state: current_term: 1 leader_uuid: "9af296bd358d4219bd49556b50ad970a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af296bd358d4219bd49556b50ad970a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34915 } } }
I20260619 08:54:56.247138 3461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9af296bd358d4219bd49556b50ad970a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af296bd358d4219bd49556b50ad970a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34915 } } }
I20260619 08:54:56.247242 3461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:56.247239 3462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a [sys.catalog]: This master's current role is: LEADER
W20260619 08:54:56.248572 3476 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:54:56.248632 3476 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:54:56.248680 3467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:54:56.248845 3467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:54:56.250236 3467 catalog_manager.cc:1347] Generated new cluster ID: 86aa50a6cd5d4f58927cd1fae7e36976
I20260619 08:54:56.250291 3467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:54:56.279690 3467 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:54:56.280306 3467 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:54:56.291581 3467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a: Generated new TSK 0
I20260619 08:54:56.291808 3467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:54:56.300139 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.1:0
--local_ip_for_outbound_sockets=127.15.208.1
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:34915
--builtin_ntp_servers=127.15.208.20:37005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:54:56.385314 3479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:56.385535 3479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:56.385558 3479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:56.386649 3479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:56.386683 3479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:56.386708 3479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.1
I20260619 08:54:56.387971 3479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.208.1
--webserver_port=0
--tserver_master_addrs=127.15.208.62:34915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.1
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:56.388154 3479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:56.388391 3479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:56.393181 3487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:56.393186 3485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:56.393186 3484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:56.393323 3479 server_base.cc:1034] running on GCE node
I20260619 08:54:56.393559 3479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:56.393806 3479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:56.394961 3479 hybrid_clock.cc:648] HybridClock initialized: now 1781859296394950 us; error 28 us; skew 500 ppm
I20260619 08:54:56.396201 3479 webserver.cc:460] Webserver started at http://127.15.208.1:41779/ using document root <none> and password file <none>
I20260619 08:54:56.396461 3479 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:56.396504 3479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:56.396661 3479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:56.397738 3479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/data/instance:
uuid: "f76d21a4ff20433daff5cbe7c0f125f6"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "035b646f5dc1405c5e17c1e4820e6618"
server_key_iv: "66438e9107692884f8c6585ffb00a97e"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.398097 3479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance:
uuid: "f76d21a4ff20433daff5cbe7c0f125f6"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "035b646f5dc1405c5e17c1e4820e6618"
server_key_iv: "66438e9107692884f8c6585ffb00a97e"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.399364 3479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:54:56.400013 3493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:56.400183 3479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:54:56.400278 3479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/wal
uuid: "f76d21a4ff20433daff5cbe7c0f125f6"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "035b646f5dc1405c5e17c1e4820e6618"
server_key_iv: "66438e9107692884f8c6585ffb00a97e"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.400358 3479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:56.426273 3479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:56.426591 3479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:56.426743 3479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:56.426982 3479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:56.427328 3479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:56.427379 3479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:56.427412 3479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:56.427451 3479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:56.433360 3479 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.1:33227
I20260619 08:54:56.433415 3606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.1:33227 every 8 connection(s)
I20260619 08:54:56.433780 3479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/data/info.pb
I20260619 08:54:56.435041 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 3479
I20260619 08:54:56.435137 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-0/wal/instance
I20260619 08:54:56.435334 16192 external_mini_cluster.cc:1428] Setting key 29714e4577eb6a76743debcea8244c32
I20260619 08:54:56.436293 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.2:0
--local_ip_for_outbound_sockets=127.15.208.2
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:34915
--builtin_ntp_servers=127.15.208.20:37005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:54:56.439769 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.434521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:46241 (local address 127.15.208.62:34915)
0619 08:54:56.434580 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.434583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.434828 (+ 245us) server_negotiation.cc:407] Connection header received
0619 08:54:56.435413 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.435415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.435433 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.435484 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:56.436311 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.436905 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.438271 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.438457 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.438519 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.439441 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:56.439442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:56.439443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:56.439448 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:56.439480 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:56.439483 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:56.439567 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.439660 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.439689 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:54:56.439981 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.434519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:34915 (local address 127.15.208.1:46241)
0619 08:54:56.434703 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.434738 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:56.435301 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:56.435572 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:56.435576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:56.435586 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:56.436217 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.436223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.436997 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.436998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.438183 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.438187 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.438547 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.438548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.438657 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.439282 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:54:56.439282 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:56.439329 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:54:56.439352 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:56.439569 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:56.439575 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:56.439578 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:56.439662 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 08:54:56.439764 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260619 08:54:56.440413 3607 heartbeater.cc:347] Connected to a master server at 127.15.208.62:34915
I20260619 08:54:56.440528 3607 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:56.440797 3607 heartbeater.cc:511] Master 127.15.208.62:34915 requested a full tablet report, sending...
I20260619 08:54:56.441321 3419 ts_manager.cc:194] Registered new tserver with Master: f76d21a4ff20433daff5cbe7c0f125f6 (127.15.208.1:33227)
I20260619 08:54:56.441931 3419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.1:46241
W20260619 08:54:56.521379 3610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:56.521577 3610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:56.521598 3610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:56.522797 3610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:56.522841 3610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:56.522866 3610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.2
I20260619 08:54:56.524278 3610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.208.2
--webserver_port=0
--tserver_master_addrs=127.15.208.62:34915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.2
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:56.524550 3610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:56.524879 3610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:56.529985 3616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:56.530012 3615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:56.530085 3618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:56.530160 3610 server_base.cc:1034] running on GCE node
I20260619 08:54:56.530380 3610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:56.530570 3610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:56.531708 3610 hybrid_clock.cc:648] HybridClock initialized: now 1781859296531681 us; error 47 us; skew 500 ppm
I20260619 08:54:56.532984 3610 webserver.cc:460] Webserver started at http://127.15.208.2:45749/ using document root <none> and password file <none>
I20260619 08:54:56.533224 3610 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:56.533269 3610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:56.533425 3610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:56.534438 3610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/data/instance:
uuid: "a3c48e0088ce4ebd90268f2e2913523b"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "61270f8c4651a5439e773344cf195cf1"
server_key_iv: "195336ad10aa69fb25f65530c2438348"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.534776 3610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance:
uuid: "a3c48e0088ce4ebd90268f2e2913523b"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "61270f8c4651a5439e773344cf195cf1"
server_key_iv: "195336ad10aa69fb25f65530c2438348"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.536058 3610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:54:56.536806 3624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:56.536963 3610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:54:56.537043 3610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/wal
uuid: "a3c48e0088ce4ebd90268f2e2913523b"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "61270f8c4651a5439e773344cf195cf1"
server_key_iv: "195336ad10aa69fb25f65530c2438348"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.537106 3610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:56.552199 3610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:56.552469 3610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:56.552575 3610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:56.552825 3610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:56.553135 3610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:56.553167 3610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:56.553189 3610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:56.553201 3610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:56.558789 3610 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.2:46713
I20260619 08:54:56.558871 3737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.2:46713 every 8 connection(s)
I20260619 08:54:56.559196 3610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/data/info.pb
I20260619 08:54:56.561486 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 3610
I20260619 08:54:56.561614 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/wal/instance
I20260619 08:54:56.561822 16192 external_mini_cluster.cc:1428] Setting key 4b0d25a66c7b8f69b45d196ee53376db
I20260619 08:54:56.562856 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.208.3:0
--local_ip_for_outbound_sockets=127.15.208.3
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:34915
--builtin_ntp_servers=127.15.208.20:37005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:54:56.564931 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.559945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.2:57609 (local address 127.15.208.62:34915)
0619 08:54:56.560007 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.560009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.560217 (+ 208us) server_negotiation.cc:407] Connection header received
0619 08:54:56.560722 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.560740 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.560769 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.560835 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:56.561698 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.562401 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.563483 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.563691 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.563763 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.564620 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:56.564621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:56.564621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:56.564626 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:56.564645 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:56.564650 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:56.564702 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.564827 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.564853 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:54:56.565129 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.559943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:34915 (local address 127.15.208.2:57609)
0619 08:54:56.560110 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.560143 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:56.560626 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:56.560927 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:56.560930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:56.560939 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:56.561590 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.561608 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.562496 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.562497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.563388 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.563390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.563772 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.563773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.563899 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.564460 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:54:56.564460 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:56.564490 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:54:56.564512 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:56.564715 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:56.564719 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:56.564738 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 08:54:56.564815 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:54:56.564919 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260619 08:54:56.565649 3738 heartbeater.cc:347] Connected to a master server at 127.15.208.62:34915
I20260619 08:54:56.565748 3738 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:56.565939 3738 heartbeater.cc:511] Master 127.15.208.62:34915 requested a full tablet report, sending...
I20260619 08:54:56.566354 3419 ts_manager.cc:194] Registered new tserver with Master: a3c48e0088ce4ebd90268f2e2913523b (127.15.208.2:46713)
I20260619 08:54:56.566916 3419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.2:57609
W20260619 08:54:56.645164 3741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:56.645391 3741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:56.645428 3741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:56.646667 3741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:56.646723 3741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:56.646826 3741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.208.3
I20260619 08:54:56.648275 3741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.208.3
--webserver_port=0
--tserver_master_addrs=127.15.208.62:34915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.208.3
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:56.648505 3741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:56.648851 3741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:56.653713 3747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:56.653726 3749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:56.653726 3746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:56.653775 3741 server_base.cc:1034] running on GCE node
I20260619 08:54:56.654130 3741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:56.654335 3741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:56.655471 3741 hybrid_clock.cc:648] HybridClock initialized: now 1781859296655459 us; error 29 us; skew 500 ppm
I20260619 08:54:56.656703 3741 webserver.cc:460] Webserver started at http://127.15.208.3:34463/ using document root <none> and password file <none>
I20260619 08:54:56.656996 3741 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:56.657043 3741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:56.657212 3741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:56.658275 3741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/data/instance:
uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "0aaba36507457a21e49e9e61e84ab7f5"
server_key_iv: "e115df16c7d092ee30fa0e3ed9e742d2"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.658622 3741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance:
uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "0aaba36507457a21e49e9e61e84ab7f5"
server_key_iv: "e115df16c7d092ee30fa0e3ed9e742d2"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.659921 3741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:54:56.660617 3755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:56.660825 3741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:54:56.660924 3741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/wal
uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef"
format_stamp: "Formatted at 2026-06-19 08:54:56 on dist-test-slave-p0h9"
server_key: "0aaba36507457a21e49e9e61e84ab7f5"
server_key_iv: "e115df16c7d092ee30fa0e3ed9e742d2"
server_key_version: "encryptionkey@0"
I20260619 08:54:56.661000 3741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:56.670166 3741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:56.670444 3741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:56.670588 3741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:56.670794 3741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:54:56.671103 3741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:54:56.671150 3741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:56.671185 3741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:54:56.671211 3741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:56.676924 3741 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.3:40531
I20260619 08:54:56.676962 3868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.3:40531 every 8 connection(s)
I20260619 08:54:56.677287 3741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/data/info.pb
I20260619 08:54:56.677980 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 3741
I20260619 08:54:56.678114 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/wal/instance
I20260619 08:54:56.678346 16192 external_mini_cluster.cc:1428] Setting key 2081894f2d6f500bceb4b44bc2609ddf
I20260619 08:54:56.682586 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.678075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.3:38161 (local address 127.15.208.62:34915)
0619 08:54:56.678132 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.678134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.678415 (+ 281us) server_negotiation.cc:407] Connection header received
0619 08:54:56.679157 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.679158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.679171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.679236 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:56.679945 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.680436 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.681439 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.681622 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.681675 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.682387 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:56.682388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:56.682388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:56.682392 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:56.682404 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:56.682408 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:56.682448 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.682505 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.682525 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:54:56.682782 3871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.678040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:34915 (local address 127.15.208.3:38161)
0619 08:54:56.678264 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.678310 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:56.679040 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:56.679316 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:56.679320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:56.679331 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:54:56.679875 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.679879 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.680506 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.680506 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.681372 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.681375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.681692 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.681693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.681798 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.682270 (+ 472us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:54:56.682270 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:54:56.682297 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:54:56.682317 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:54:56.682448 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:54:56.682450 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:54:56.682453 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:56.682497 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 08:54:56.682581 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260619 08:54:56.683336 3869 heartbeater.cc:347] Connected to a master server at 127.15.208.62:34915
I20260619 08:54:56.683411 3869 heartbeater.cc:464] Registering TS with master...
I20260619 08:54:56.683565 3869 heartbeater.cc:511] Master 127.15.208.62:34915 requested a full tablet report, sending...
I20260619 08:54:56.683871 3419 ts_manager.cc:194] Registered new tserver with Master: fa4b9ec5ca6d432d8c0466c5f2a3acef (127.15.208.3:40531)
I20260619 08:54:56.684237 3419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.208.3:38161
I20260619 08:54:56.690062 16192 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:54:56.693938 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.691424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45802 (local address 127.15.208.62:34915)
0619 08:54:56.691485 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.691487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.691566 (+ 79us) server_negotiation.cc:407] Connection header received
0619 08:54:56.691612 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.691613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.691621 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.691670 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:56.691787 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.692228 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.692971 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.693149 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.693199 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.693730 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:56.693731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:56.693731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:56.693735 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:56.693746 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:56.693750 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:56.693795 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.693828 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.693846 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:54:56.695312 3419 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:54:56.696834 3419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:54:56.703248 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.698901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:33227 (local address 127.0.0.1:49070)
0619 08:54:56.699019 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.699057 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:56.699123 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:56.699648 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:56.699649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:56.699651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:56.699696 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.699697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.701116 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.701117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.702145 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.702147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.702997 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.702998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.703054 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.703057 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:54:56.703105 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:54:56.703134 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":72,"threads_started":1}
I20260619 08:54:56.703305 3881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.699287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49070 (local address 127.15.208.1:33227)
0619 08:54:56.699441 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.699447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.699458 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:54:56.699497 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.699499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.699514 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.699776 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:56.699843 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.701012 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.702228 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.702913 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.702993 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.703090 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.703121 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.703166 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260619 08:54:56.703413 3879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.699014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.15.208.2:46713)
0619 08:54:56.699204 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.699209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.699219 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:54:56.699257 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.699259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.699273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.699366 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:56.699529 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.700459 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.701959 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.702697 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.702776 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.702868 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.703264 (+ 396us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.703309 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260619 08:54:56.703145 3878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.698917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:46713 (local address 127.0.0.1:37384)
0619 08:54:56.699103 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.699113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:56.699163 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:56.699376 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:56.699379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:56.699381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:56.699444 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.699445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.700579 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.700579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.701874 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.701876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.702775 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.702776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.702893 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.702910 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 08:54:56.702998 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:54:56.703049 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":119,"threads_started":1}
I20260619 08:54:56.704078 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.699123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:40531 (local address 127.0.0.1:39564)
0619 08:54:56.699245 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.699254 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:56.699293 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:56.699877 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:56.699878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:56.699880 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:56.699917 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.699918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.701181 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.701182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.702341 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.702344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.703233 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.703234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.703929 (+ 695us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.703934 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:54:56.703973 (+ 39us) client_negotiation.cc:240] Negotiation successful
0619 08:54:56.704012 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":83,"threads_started":1}
I20260619 08:54:56.704052 3541 tablet_service.cc:1467] Processing CreateTablet for tablet 53fa45d350114dde8aecdb955c8812ad (DEFAULT_TABLE table=test-table [id=91531d93659846d1b42a06a04727ec97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:56.704192 3672 tablet_service.cc:1467] Processing CreateTablet for tablet 53fa45d350114dde8aecdb955c8812ad (DEFAULT_TABLE table=test-table [id=91531d93659846d1b42a06a04727ec97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:56.704424 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.699532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.15.208.3:40531)
0619 08:54:56.699728 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.699733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.699741 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:54:56.699777 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.699778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.699792 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.699864 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:56.699967 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.701087 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.702429 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.703118 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.704193 (+ 1075us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.704279 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.704304 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.704344 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
I20260619 08:54:56.704452 3541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53fa45d350114dde8aecdb955c8812ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:56.704540 3672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53fa45d350114dde8aecdb955c8812ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:56.705428 3803 tablet_service.cc:1467] Processing CreateTablet for tablet 53fa45d350114dde8aecdb955c8812ad (DEFAULT_TABLE table=test-table [id=91531d93659846d1b42a06a04727ec97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:54:56.705756 3803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53fa45d350114dde8aecdb955c8812ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:56.707247 3883 tablet_bootstrap.cc:492] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6: Bootstrap starting.
I20260619 08:54:56.707877 3884 tablet_bootstrap.cc:492] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: Bootstrap starting.
I20260619 08:54:56.707952 3883 tablet_bootstrap.cc:654] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:56.708292 3883 log.cc:824] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:56.708534 3884 tablet_bootstrap.cc:654] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:56.708885 3884 log.cc:824] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:56.709044 3886 tablet_bootstrap.cc:492] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: Bootstrap starting.
I20260619 08:54:56.709211 3883 tablet_bootstrap.cc:492] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6: No bootstrap required, opened a new log
I20260619 08:54:56.709304 3883 ts_tablet_manager.cc:1397] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:54:56.709702 3886 tablet_bootstrap.cc:654] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:56.709986 3886 log.cc:824] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:56.710736 3886 tablet_bootstrap.cc:492] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: No bootstrap required, opened a new log
I20260619 08:54:56.710807 3886 ts_tablet_manager.cc:1397] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:54:56.711738 3883 raft_consensus.cc:348] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33227 } } peers { permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 } } peers { permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 } }
I20260619 08:54:56.711915 3883 raft_consensus.cc:374] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:56.711954 3883 raft_consensus.cc:729] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76d21a4ff20433daff5cbe7c0f125f6, State: Initialized, Role: FOLLOWER
I20260619 08:54:56.712076 3883 consensus_queue.cc:260] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33227 } } peers { permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 } } peers { permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 } }
I20260619 08:54:56.712350 3883 ts_tablet_manager.cc:1428] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:54:56.712384 3607 heartbeater.cc:503] Master 127.15.208.62:34915 was elected leader, sending a full tablet report...
I20260619 08:54:56.713110 3884 tablet_bootstrap.cc:492] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: No bootstrap required, opened a new log
I20260619 08:54:56.713182 3884 ts_tablet_manager.cc:1397] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 08:54:56.713475 3886 raft_consensus.cc:348] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33227 } } peers { permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 } } peers { permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 } }
I20260619 08:54:56.713646 3886 raft_consensus.cc:374] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:56.713696 3886 raft_consensus.cc:729] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4b9ec5ca6d432d8c0466c5f2a3acef, State: Initialized, Role: FOLLOWER
I20260619 08:54:56.713814 3886 consensus_queue.cc:260] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33227 } } peers { permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 } } peers { permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 } }
I20260619 08:54:56.714115 3886 ts_tablet_manager.cc:1428] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 08:54:56.714134 3869 heartbeater.cc:503] Master 127.15.208.62:34915 was elected leader, sending a full tablet report...
I20260619 08:54:56.715665 3884 raft_consensus.cc:348] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33227 } } peers { permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 } } peers { permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 } }
I20260619 08:54:56.715790 3884 raft_consensus.cc:374] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:56.715812 3884 raft_consensus.cc:729] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c48e0088ce4ebd90268f2e2913523b, State: Initialized, Role: FOLLOWER
I20260619 08:54:56.715909 3884 consensus_queue.cc:260] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33227 } } peers { permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 } } peers { permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 } }
I20260619 08:54:56.716136 3884 ts_tablet_manager.cc:1428] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:54:56.716198 3738 heartbeater.cc:503] Master 127.15.208.62:34915 was elected leader, sending a full tablet report...
I20260619 08:54:56.717370 3889 raft_consensus.cc:482] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:54:56.717448 3889 raft_consensus.cc:504] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33227 } } peers { permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 } } peers { permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 } }
I20260619 08:54:56.717795 3889 leader_election.cc:288] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3c48e0088ce4ebd90268f2e2913523b (127.15.208.2:46713), fa4b9ec5ca6d432d8c0466c5f2a3acef (127.15.208.3:40531)
I20260619 08:54:56.721474 3879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.718009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:34473 (local address 127.15.208.2:46713)
0619 08:54:56.718054 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.718056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.718064 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:54:56.718092 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.718093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.718100 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.718165 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:56.718305 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.719057 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.720525 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.721111 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.721189 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.721196 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.721366 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.721404 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:54:56.721474 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.718035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.208.1:35505 (local address 127.15.208.3:40531)
0619 08:54:56.718076 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.718079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.718167 (+ 88us) server_negotiation.cc:407] Connection header received
0619 08:54:56.718208 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.718209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.718216 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.718262 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:54:56.718386 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.719057 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.720525 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.721114 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.721188 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.721196 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.721366 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.721404 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:54:56.721487 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.717915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.2:46713 (local address 127.15.208.1:34473)
0619 08:54:56.717950 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.717963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:56.718006 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:56.718171 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:56.718172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:56.718174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:56.718239 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.718241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.719141 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.719142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.720441 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.720444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.721199 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.721200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.721282 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.721287 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:54:56.721352 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 08:54:56.721406 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":40}
I20260619 08:54:56.721494 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.717949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.3:40531 (local address 127.15.208.1:35505)
0619 08:54:56.718102 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.718110 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:54:56.718155 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:54:56.718266 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:54:56.718267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:54:56.718268 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:54:56.718322 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.718323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.719141 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.719142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.720441 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:54:56.720444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.721199 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.721200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:54:56.721293 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.721297 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:54:56.721352 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:54:56.721401 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":79,"thread_start_us":77,"threads_started":1}
I20260619 08:54:56.721861 3692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53fa45d350114dde8aecdb955c8812ad" candidate_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c48e0088ce4ebd90268f2e2913523b" is_pre_election: true
I20260619 08:54:56.721994 3692 raft_consensus.cc:2454] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76d21a4ff20433daff5cbe7c0f125f6 in term 0.
I20260619 08:54:56.722179 3823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53fa45d350114dde8aecdb955c8812ad" candidate_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" is_pre_election: true
I20260619 08:54:56.722229 3497 leader_election.cc:302] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3c48e0088ce4ebd90268f2e2913523b, f76d21a4ff20433daff5cbe7c0f125f6; no voters:
I20260619 08:54:56.722312 3823 raft_consensus.cc:2454] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76d21a4ff20433daff5cbe7c0f125f6 in term 0.
I20260619 08:54:56.722342 3889 raft_consensus.cc:2781] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:54:56.722393 3889 raft_consensus.cc:482] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:54:56.722427 3889 raft_consensus.cc:3037] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:56.723278 3889 raft_consensus.cc:504] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33227 } } peers { permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 } } peers { permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 } }
I20260619 08:54:56.723419 3889 leader_election.cc:288] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [CANDIDATE]: Term 1 election: Requested vote from peers a3c48e0088ce4ebd90268f2e2913523b (127.15.208.2:46713), fa4b9ec5ca6d432d8c0466c5f2a3acef (127.15.208.3:40531)
I20260619 08:54:56.723585 3823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53fa45d350114dde8aecdb955c8812ad" candidate_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef"
I20260619 08:54:56.723577 3692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53fa45d350114dde8aecdb955c8812ad" candidate_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c48e0088ce4ebd90268f2e2913523b"
I20260619 08:54:56.723667 3823 raft_consensus.cc:3037] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:56.723663 3692 raft_consensus.cc:3037] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:56.724514 3692 raft_consensus.cc:2454] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76d21a4ff20433daff5cbe7c0f125f6 in term 1.
I20260619 08:54:56.724676 3497 leader_election.cc:302] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3c48e0088ce4ebd90268f2e2913523b, f76d21a4ff20433daff5cbe7c0f125f6; no voters:
I20260619 08:54:56.724697 3823 raft_consensus.cc:2454] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76d21a4ff20433daff5cbe7c0f125f6 in term 1.
I20260619 08:54:56.724777 3889 raft_consensus.cc:2781] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:56.724931 3889 raft_consensus.cc:686] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 1 LEADER]: Becoming Leader. State: Replica: f76d21a4ff20433daff5cbe7c0f125f6, State: Running, Role: LEADER
I20260619 08:54:56.725054 3889 consensus_queue.cc:237] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33227 } } peers { permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 } } peers { permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 } }
I20260619 08:54:56.725901 3419 catalog_manager.cc:5495] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to f76d21a4ff20433daff5cbe7c0f125f6 (127.15.208.1). New cstate: current_term: 1 leader_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76d21a4ff20433daff5cbe7c0f125f6" member_type: VOTER last_known_addr { host: "127.15.208.1" port: 33227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 } health_report { overall_health: UNKNOWN } } }
I20260619 08:54:56.735946 3881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.733975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49072 (local address 127.15.208.1:33227)
0619 08:54:56.734035 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:56.734036 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:56.734045 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:54:56.734091 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:56.734092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:56.734106 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:56.734167 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:54:56.734257 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.734709 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.735360 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:56.735531 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:56.735579 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:56.735695 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:54:56.735781 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:54:56.735837 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:56.735865 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 08:54:56.735880 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:54:56.737629 3692 raft_consensus.cc:1261] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b [term 1 FOLLOWER]: Refusing update from remote peer f76d21a4ff20433daff5cbe7c0f125f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:56.737630 3823 raft_consensus.cc:1261] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef [term 1 FOLLOWER]: Refusing update from remote peer f76d21a4ff20433daff5cbe7c0f125f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:54:56.737967 3893 consensus_queue.cc:1035] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3c48e0088ce4ebd90268f2e2913523b" member_type: VOTER last_known_addr { host: "127.15.208.2" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:56.738128 3889 consensus_queue.cc:1035] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4b9ec5ca6d432d8c0466c5f2a3acef" member_type: VOTER last_known_addr { host: "127.15.208.3" port: 40531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:54:56.741189 3898 mvcc.cc:205] Tried to move back new op lower bound from 7298495679434919936 to 7298495679386525696. Current Snapshot: MvccSnapshot[applied={T|T < 7298495679434919936 or (T in {7298495679434919936})}]
I20260619 08:54:56.742646 3417 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:56.742770 3417 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:54:56.743506 3417 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9af296bd358d4219bd49556b50ad970a: Sending DeleteTablet for 3 replicas of tablet 53fa45d350114dde8aecdb955c8812ad
I20260619 08:54:56.743845 3541 tablet_service.cc:1514] Processing DeleteTablet for tablet 53fa45d350114dde8aecdb955c8812ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:56 UTC) from {username='slave'} at 127.0.0.1:49070
I20260619 08:54:56.743860 3672 tablet_service.cc:1514] Processing DeleteTablet for tablet 53fa45d350114dde8aecdb955c8812ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:56 UTC) from {username='slave'} at 127.0.0.1:37384
I20260619 08:54:56.743861 3803 tablet_service.cc:1514] Processing DeleteTablet for tablet 53fa45d350114dde8aecdb955c8812ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:54:56 UTC) from {username='slave'} at 127.0.0.1:39564
I20260619 08:54:56.744154 3905 tablet_replica.cc:330] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6: stopping tablet replica
I20260619 08:54:56.744256 3905 raft_consensus.cc:2229] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:54:56.744345 3905 raft_consensus.cc:2258] T 53fa45d350114dde8aecdb955c8812ad P f76d21a4ff20433daff5cbe7c0f125f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:56.744412 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 3479
I20260619 08:54:56.745621 3907 tablet_replica.cc:330] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: stopping tablet replica
I20260619 08:54:56.745733 3907 raft_consensus.cc:2229] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:56.745846 3907 raft_consensus.cc:2258] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:56.746024 3906 tablet_replica.cc:330] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: stopping tablet replica
I20260619 08:54:56.746140 3906 raft_consensus.cc:2229] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:54:56.746261 3906 raft_consensus.cc:2258] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:54:56.746367 3907 ts_tablet_manager.cc:1905] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:56.747311 3906 ts_tablet_manager.cc:1905] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:54:56.748004 3907 ts_tablet_manager.cc:1918] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:56.748093 3907 log.cc:1196] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-2/wal/wals/53fa45d350114dde8aecdb955c8812ad
I20260619 08:54:56.748344 3907 ts_tablet_manager.cc:1939] T 53fa45d350114dde8aecdb955c8812ad P fa4b9ec5ca6d432d8c0466c5f2a3acef: Deleting consensus metadata
I20260619 08:54:56.748781 3405 catalog_manager.cc:4841] TS fa4b9ec5ca6d432d8c0466c5f2a3acef (127.15.208.3:40531): tablet 53fa45d350114dde8aecdb955c8812ad (table test-table [id=91531d93659846d1b42a06a04727ec97]) successfully deleted
I20260619 08:54:56.749289 3906 ts_tablet_manager.cc:1918] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:54:56.749367 3906 log.cc:1196] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: Deleting WAL directory at /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781859057751076-16192-0/minicluster-data/ts-1/wal/wals/53fa45d350114dde8aecdb955c8812ad
I20260619 08:54:56.749619 3906 ts_tablet_manager.cc:1939] T 53fa45d350114dde8aecdb955c8812ad P a3c48e0088ce4ebd90268f2e2913523b: Deleting consensus metadata
I20260619 08:54:56.749998 3406 catalog_manager.cc:4841] TS a3c48e0088ce4ebd90268f2e2913523b (127.15.208.2:46713): tablet 53fa45d350114dde8aecdb955c8812ad (table test-table [id=91531d93659846d1b42a06a04727ec97]) successfully deleted
W20260619 08:54:56.751003 3405 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.208.1:33227 (error 108)
I20260619 08:54:56.751441 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:56.751318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.1:33227 (local address 127.0.0.1:49082)
0619 08:54:56.751368 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 08:54:56.751408 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.1:33227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 08:54:56.751457 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 3610
W20260619 08:54:56.751570 3405 catalog_manager.cc:4584] TS f76d21a4ff20433daff5cbe7c0f125f6 (127.15.208.1:33227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 53fa45d350114dde8aecdb955c8812ad: Network error: Client connection negotiation failed: client connection to 127.15.208.1:33227: connect: Connection refused (error 111)
I20260619 08:54:56.756120 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 3741
I20260619 08:54:56.760530 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 3388
2026-06-19T08:54:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (662 ms)
[----------] 6 tests from EncryptionPolicyTest (8215 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:54:56 dist-test-slave-p0h9 krb5kdc[3911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:54:56 dist-test-slave-p0h9 krb5kdc[3911](info): set up 2 sockets
Jun 19 08:54:56 dist-test-slave-p0h9 krb5kdc[3911](info): commencing operation
krb5kdc: starting...
W20260619 08:54:57.805275 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:54:57 dist-test-slave-p0h9 krb5kdc[3911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:54:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:54:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:54:57.921756 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39257
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:58.002583 3924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:58.002781 3924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:58.002804 3924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:58.003978 3924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:54:58.004021 3924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:58.004036 3924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:58.004089 3924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:54:58.004108 3924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:54:58.005544 3924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39257
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:58.005789 3924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:58.006028 3924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:58.010887 3929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:58.010855 3932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:58.010908 3930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:58.011111 3924 server_base.cc:1034] running on GCE node
I20260619 08:54:58.011315 3924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:58.011616 3924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:58.012789 3924 hybrid_clock.cc:648] HybridClock initialized: now 1781859298012770 us; error 37 us; skew 500 ppm
Jun 19 08:54:58 dist-test-slave-p0h9 krb5kdc[3911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859298, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:58.015286 3924 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:54:58.015743 3924 webserver.cc:460] Webserver started at http://127.0.0.1:32869/ using document root <none> and password file <none>
I20260619 08:54:58.015957 3924 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:58.016001 3924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:58.016129 3924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:58.017195 3924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "e85357ecf740428f9913dedc14a04c34"
format_stamp: "Formatted at 2026-06-19 08:54:58 on dist-test-slave-p0h9"
server_key: "2587d6da4566c128b04bd585448d4d08"
server_key_iv: "5da87feb76ec04b6ecf1aceb5f26fe7c"
server_key_version: "encryptionkey@0"
I20260619 08:54:58.017488 3924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "e85357ecf740428f9913dedc14a04c34"
format_stamp: "Formatted at 2026-06-19 08:54:58 on dist-test-slave-p0h9"
server_key: "2587d6da4566c128b04bd585448d4d08"
server_key_iv: "5da87feb76ec04b6ecf1aceb5f26fe7c"
server_key_version: "encryptionkey@0"
I20260619 08:54:58.018748 3924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:54:58.019383 3939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:58.019582 3924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:54:58.019671 3924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "e85357ecf740428f9913dedc14a04c34"
format_stamp: "Formatted at 2026-06-19 08:54:58 on dist-test-slave-p0h9"
server_key: "2587d6da4566c128b04bd585448d4d08"
server_key_iv: "5da87feb76ec04b6ecf1aceb5f26fe7c"
server_key_version: "encryptionkey@0"
I20260619 08:54:58.019737 3924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:58.038442 3924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:58.040630 3924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:58.040890 3924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:58.045339 3924 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39257
I20260619 08:54:58.045387 3991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39257 every 8 connection(s)
I20260619 08:54:58.045794 3924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:54:58.046610 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 3924
I20260619 08:54:58.046644 3992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:58.046715 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:54:58.046967 16192 external_mini_cluster.cc:1428] Setting key 0fadfcf06f4ceb029a61ffaf6ea76722
I20260619 08:54:58.049638 3992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34: Bootstrap starting.
I20260619 08:54:58.050252 3992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:58.050529 3992 log.cc:824] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:58.051235 3992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34: No bootstrap required, opened a new log
I20260619 08:54:58.053406 3992 raft_consensus.cc:348] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85357ecf740428f9913dedc14a04c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39257 } }
I20260619 08:54:58.053575 3992 raft_consensus.cc:374] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:58.053620 3992 raft_consensus.cc:729] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e85357ecf740428f9913dedc14a04c34, State: Initialized, Role: FOLLOWER
I20260619 08:54:58.053757 3992 consensus_queue.cc:260] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85357ecf740428f9913dedc14a04c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39257 } }
I20260619 08:54:58.053838 3992 raft_consensus.cc:388] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:54:58.053869 3992 raft_consensus.cc:482] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Jun 19 08:54:58 dist-test-slave-p0h9 krb5kdc[3911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:54:58.053915 3992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:58.054461 3992 raft_consensus.cc:504] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85357ecf740428f9913dedc14a04c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39257 } }
I20260619 08:54:58.054560 3992 leader_election.cc:302] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e85357ecf740428f9913dedc14a04c34; no voters:
I20260619 08:54:58.054751 3992 leader_election.cc:288] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:54:58.054804 3997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:58.054965 3997 raft_consensus.cc:686] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [term 1 LEADER]: Becoming Leader. State: Replica: e85357ecf740428f9913dedc14a04c34, State: Running, Role: LEADER
I20260619 08:54:58.055047 3992 sys_catalog.cc:564] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:54:58.055071 3997 consensus_queue.cc:237] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85357ecf740428f9913dedc14a04c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39257 } }
I20260619 08:54:58.055457 3998 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e85357ecf740428f9913dedc14a04c34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85357ecf740428f9913dedc14a04c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39257 } } }
I20260619 08:54:58.055612 3998 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:58.055689 3999 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e85357ecf740428f9913dedc14a04c34. Latest consensus state: current_term: 1 leader_uuid: "e85357ecf740428f9913dedc14a04c34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85357ecf740428f9913dedc14a04c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39257 } } }
I20260619 08:54:58.055768 3999 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34 [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:58.055845 4004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:54:58.056321 4004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:54:58.057857 4004 catalog_manager.cc:1347] Generated new cluster ID: e50b1728d780431faf829b45425fa9e5
I20260619 08:54:58.057911 4004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:54:58.059088 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:58.047554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60609 (local address 127.0.0.1:39257)
0619 08:54:58.047751 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:58.047762 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:58.047798 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:54:58.048243 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:58.048250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:58.048298 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:58.048449 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:58.049151 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:58.050086 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:58.051151 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:58.051600 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:58.051711 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:58.054422 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:58.054426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:58.054426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:58.054450 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:58.057289 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:58.057839 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:58.057843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:58.057844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:58.057944 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:58.058200 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:58.058202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:58.058202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:58.058493 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:58.058647 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:58.058798 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 08:54:58.058890 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260619 08:54:58.060030 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:54:58.069080 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:58.061443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48975 (local address 127.0.0.1:39257)
0619 08:54:58.061475 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:58.061476 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:58.061576 (+ 100us) server_negotiation.cc:407] Connection header received
0619 08:54:58.061637 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:58.061638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:58.061646 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:58.061701 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:58.062333 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:58.063324 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:58.064351 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:58.064667 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:58.064710 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:58.066253 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:58.066254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:58.066255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:54:58.066261 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:58.067810 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:58.068248 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:58.068249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:58.068249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:58.068305 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:54:58.068536 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:54:58.068536 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:54:58.068537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:54:58.068648 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:58.068758 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:58.068930 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 08:54:58.068988 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 08:54:58.074326 4004 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:54:58.074831 4004 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:54:58.081749 4004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e85357ecf740428f9913dedc14a04c34: Generated new TSK 0
I20260619 08:54:58.081948 4004 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:54:58.086997 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 3924
W20260619 08:54:58.091602 3909 connection.cc:535] client connection to 127.0.0.1:39257 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39257): Connection reset by peer (error 104)
2026-06-19T08:54:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:54:58 dist-test-slave-p0h9 krb5kdc[4023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:54:58 dist-test-slave-p0h9 krb5kdc[4023](info): set up 2 sockets
Jun 19 08:54:58 dist-test-slave-p0h9 krb5kdc[4023](info): commencing operation
krb5kdc: starting...
W20260619 08:54:59.117398 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:54:59 dist-test-slave-p0h9 krb5kdc[4023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:54:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:54:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:54:59.231482 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:54:59.310278 4036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:54:59.310500 4036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:54:59.310523 4036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:54:59.311628 4036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:54:59.311662 4036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:54:59.311677 4036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:54:59.311689 4036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:54:59.311702 4036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:54:59.313131 4036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34761
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:54:59.313316 4036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:54:59.313488 4036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:54:59.318267 4042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:59.318248 4041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:54:59.318264 4044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:54:59.318490 4036 server_base.cc:1034] running on GCE node
I20260619 08:54:59.318734 4036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:54:59.319026 4036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:54:59.320221 4036 hybrid_clock.cc:648] HybridClock initialized: now 1781859299320205 us; error 40 us; skew 500 ppm
Jun 19 08:54:59 dist-test-slave-p0h9 krb5kdc[4023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859299, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:54:59.326426 4036 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:54:59.326942 4036 webserver.cc:460] Webserver started at http://127.0.0.1:37405/ using document root <none> and password file <none>
I20260619 08:54:59.327174 4036 fs_manager.cc:359] Metadata directory not provided
I20260619 08:54:59.327239 4036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:54:59.327385 4036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:54:59.328462 4036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "71734e1ae09d45c789b0208e4bc93f29"
format_stamp: "Formatted at 2026-06-19 08:54:59 on dist-test-slave-p0h9"
server_key: "10714ca8e3756ec2825268ea31a21627"
server_key_iv: "da7fd1018c6117753852ea9da5dc43ad"
server_key_version: "encryptionkey@0"
I20260619 08:54:59.328886 4036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "71734e1ae09d45c789b0208e4bc93f29"
format_stamp: "Formatted at 2026-06-19 08:54:59 on dist-test-slave-p0h9"
server_key: "10714ca8e3756ec2825268ea31a21627"
server_key_iv: "da7fd1018c6117753852ea9da5dc43ad"
server_key_version: "encryptionkey@0"
I20260619 08:54:59.330235 4036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:54:59.330968 4051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:54:59.331139 4036 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:54:59.331236 4036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "71734e1ae09d45c789b0208e4bc93f29"
format_stamp: "Formatted at 2026-06-19 08:54:59 on dist-test-slave-p0h9"
server_key: "10714ca8e3756ec2825268ea31a21627"
server_key_iv: "da7fd1018c6117753852ea9da5dc43ad"
server_key_version: "encryptionkey@0"
I20260619 08:54:59.331317 4036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:54:59.352658 4036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:54:59.354589 4036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:54:59.354755 4036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:54:59.358973 4036 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34761
I20260619 08:54:59.359010 4103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34761 every 8 connection(s)
I20260619 08:54:59.359390 4036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:54:59.360249 4104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:54:59.362670 4104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29: Bootstrap starting.
I20260619 08:54:59.363272 4104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29: Neither blocks nor log segments found. Creating new log.
I20260619 08:54:59.363590 4104 log.cc:824] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29: Log is configured to *not* fsync() on all Append() calls
I20260619 08:54:59.364307 4104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29: No bootstrap required, opened a new log
I20260619 08:54:59.365958 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 4036
I20260619 08:54:59.366092 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:54:59.366016 4104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71734e1ae09d45c789b0208e4bc93f29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34761 } }
I20260619 08:54:59.366235 4104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:54:59.366286 4104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71734e1ae09d45c789b0208e4bc93f29, State: Initialized, Role: FOLLOWER
I20260619 08:54:59.366300 16192 external_mini_cluster.cc:1428] Setting key 3a5b6682c95f44e8a87842c01b883c0d
I20260619 08:54:59.366436 4104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71734e1ae09d45c789b0208e4bc93f29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34761 } }
I20260619 08:54:59.366554 4104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:54:59.366591 4104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:54:59.366645 4104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:54:59.368031 4104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71734e1ae09d45c789b0208e4bc93f29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34761 } }
I20260619 08:54:59.368142 4104 leader_election.cc:302] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71734e1ae09d45c789b0208e4bc93f29; no voters:
I20260619 08:54:59.368301 4104 leader_election.cc:288] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:54:59.368353 4109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:54:59.368512 4109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [term 1 LEADER]: Becoming Leader. State: Replica: 71734e1ae09d45c789b0208e4bc93f29, State: Running, Role: LEADER
I20260619 08:54:59.368548 4104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:54:59.368688 4109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71734e1ae09d45c789b0208e4bc93f29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34761 } }
I20260619 08:54:59.369099 4109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71734e1ae09d45c789b0208e4bc93f29. Latest consensus state: current_term: 1 leader_uuid: "71734e1ae09d45c789b0208e4bc93f29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71734e1ae09d45c789b0208e4bc93f29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34761 } } }
I20260619 08:54:59.369230 4109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:59.369449 4118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:54:59.369927 4118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:54:59.370330 4111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71734e1ae09d45c789b0208e4bc93f29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71734e1ae09d45c789b0208e4bc93f29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34761 } } }
I20260619 08:54:59.370412 4111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29 [sys.catalog]: This master's current role is: LEADER
I20260619 08:54:59.371577 4118 catalog_manager.cc:1347] Generated new cluster ID: 3a081fc4a322404a9901aa8d2f486ee5
I20260619 08:54:59.371629 4118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:54:59.372455 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:59.366799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50641 (local address 127.0.0.1:34761)
0619 08:54:59.366977 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:59.366987 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:59.367020 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:54:59.367154 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:59.367161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:59.367233 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:59.367396 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:59.367548 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:59.368981 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:59.370153 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:59.370756 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:59.370883 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:59.371594 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:59.371598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:59.371598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:59.371826 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:59.371862 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:59.371876 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:59.371954 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:59.372216 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 08:54:59.372244 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260619 08:54:59.373461 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:54:59.380635 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:54:59.377079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53147 (local address 127.0.0.1:34761)
0619 08:54:59.377135 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:54:59.377136 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:54:59.377309 (+ 173us) server_negotiation.cc:407] Connection header received
0619 08:54:59.377380 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:54:59.377381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:54:59.377391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:54:59.377440 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:54:59.377582 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:59.378275 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:59.379285 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:54:59.379575 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:54:59.379634 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:54:59.380389 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:54:59.380390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:54:59.380391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:54:59.380395 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:54:59.380409 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:54:59.380413 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:54:59.380461 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:54:59.380543 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 08:54:59.380556 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:54:59.386197 4118 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:54:59.386919 4118 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:54:59.389734 4118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71734e1ae09d45c789b0208e4bc93f29: Generated new TSK 0
I20260619 08:54:59.389906 4118 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:54:59.391692 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 4036
W20260619 08:54:59.395650 4021 connection.cc:535] client connection to 127.0.0.1:34761 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34761): Connection reset by peer (error 104)
2026-06-19T08:54:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:54:59 dist-test-slave-p0h9 krb5kdc[4135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:54:59 dist-test-slave-p0h9 krb5kdc[4135](info): set up 2 sockets
Jun 19 08:54:59 dist-test-slave-p0h9 krb5kdc[4135](info): commencing operation
krb5kdc: starting...
W20260619 08:55:00.441334 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:00 dist-test-slave-p0h9 krb5kdc[4135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:55:00.556571 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33591
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:00.637499 4148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:00.637705 4148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:00.637730 4148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:00.638845 4148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:00.638890 4148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:00.638903 4148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:00.638947 4148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:00.638971 4148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:00.640399 4148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33591
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:00.640635 4148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:00.640866 4148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:00.646111 4156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:00.646131 4153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:00.646126 4154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:00.646178 4148 server_base.cc:1034] running on GCE node
I20260619 08:55:00.646574 4148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:00.646871 4148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:00.648015 4148 hybrid_clock.cc:648] HybridClock initialized: now 1781859300647997 us; error 38 us; skew 500 ppm
Jun 19 08:55:00 dist-test-slave-p0h9 krb5kdc[4135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859300, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:00.650504 4148 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:55:00.650959 4148 webserver.cc:460] Webserver started at http://127.0.0.1:35985/ using document root <none> and password file <none>
I20260619 08:55:00.651185 4148 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:00.651252 4148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:00.651394 4148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:00.652456 4148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "7c82c809d2a94ea0ba86d204b195ff21"
format_stamp: "Formatted at 2026-06-19 08:55:00 on dist-test-slave-p0h9"
server_key: "7ffdf1621bf957e0aede5e695fe4009d"
server_key_iv: "e7832bcb8a1d133de766c335f2c02169"
server_key_version: "encryptionkey@0"
I20260619 08:55:00.652870 4148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "7c82c809d2a94ea0ba86d204b195ff21"
format_stamp: "Formatted at 2026-06-19 08:55:00 on dist-test-slave-p0h9"
server_key: "7ffdf1621bf957e0aede5e695fe4009d"
server_key_iv: "e7832bcb8a1d133de766c335f2c02169"
server_key_version: "encryptionkey@0"
I20260619 08:55:00.654177 4148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:55:00.654855 4163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:00.655020 4148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:55:00.655120 4148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "7c82c809d2a94ea0ba86d204b195ff21"
format_stamp: "Formatted at 2026-06-19 08:55:00 on dist-test-slave-p0h9"
server_key: "7ffdf1621bf957e0aede5e695fe4009d"
server_key_iv: "e7832bcb8a1d133de766c335f2c02169"
server_key_version: "encryptionkey@0"
I20260619 08:55:00.655198 4148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:00.658512 4148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:00.660336 4148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:00.660493 4148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:00.664575 4148 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33591
I20260619 08:55:00.664608 4215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33591 every 8 connection(s)
I20260619 08:55:00.664988 4148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:00.665783 4216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:00.668038 4216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21: Bootstrap starting.
I20260619 08:55:00.668608 4216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:00.668918 4216 log.cc:824] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:00.669566 4216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21: No bootstrap required, opened a new log
I20260619 08:55:00.670859 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 4148
I20260619 08:55:00.671015 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:00.671239 16192 external_mini_cluster.cc:1428] Setting key 55d7db4831d37dca84f4744375ce2ab7
I20260619 08:55:00.671623 4216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c82c809d2a94ea0ba86d204b195ff21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33591 } }
I20260619 08:55:00.671826 4216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:00.671921 4216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c82c809d2a94ea0ba86d204b195ff21, State: Initialized, Role: FOLLOWER
I20260619 08:55:00.672076 4216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c82c809d2a94ea0ba86d204b195ff21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33591 } }
I20260619 08:55:00.672158 4216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:00.672189 4216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:00.672236 4216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:00.673147 4216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c82c809d2a94ea0ba86d204b195ff21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33591 } }
I20260619 08:55:00.673264 4216 leader_election.cc:302] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c82c809d2a94ea0ba86d204b195ff21; no voters:
I20260619 08:55:00.673497 4216 leader_election.cc:288] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:00.673552 4221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:00.673592 4220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:00.671758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37095 (local address 127.0.0.1:33591)
0619 08:55:00.671988 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:00.671998 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:00.672029 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:55:00.672143 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:00.672150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:00.672181 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:00.672341 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:00.673073 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:00.673076 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:00.673076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:00.673081 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:00.673130 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:00.673146 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:00.673231 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:00.673330 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 08:55:00.673374 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
I20260619 08:55:00.673727 4221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [term 1 LEADER]: Becoming Leader. State: Replica: 7c82c809d2a94ea0ba86d204b195ff21, State: Running, Role: LEADER
I20260619 08:55:00.673774 4216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:00.673911 4221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c82c809d2a94ea0ba86d204b195ff21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33591 } }
I20260619 08:55:00.674995 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:00.675575 4223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c82c809d2a94ea0ba86d204b195ff21. Latest consensus state: current_term: 1 leader_uuid: "7c82c809d2a94ea0ba86d204b195ff21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c82c809d2a94ea0ba86d204b195ff21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33591 } } }
I20260619 08:55:00.675796 4223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:00.675863 4222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c82c809d2a94ea0ba86d204b195ff21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c82c809d2a94ea0ba86d204b195ff21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33591 } } }
I20260619 08:55:00.675917 4222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21 [sys.catalog]: This master's current role is: LEADER
W20260619 08:55:00.675792 4236 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:55:00.676118 4236 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:55:00.676216 4238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:00.676430 4238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:00.677856 4220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:00.676538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34747 (local address 127.0.0.1:33591)
0619 08:55:00.676574 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:00.676576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:00.676702 (+ 126us) server_negotiation.cc:407] Connection header received
0619 08:55:00.676775 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:00.676776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:00.676781 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:00.676844 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:00.677654 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:00.677655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:00.677656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:00.677660 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:00.677674 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:00.677678 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:00.677727 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:00.677778 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 08:55:00.677793 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:55:00.678218 4238 catalog_manager.cc:1347] Generated new cluster ID: 962f0b0436e54e1898111a9d455b961f
I20260619 08:55:00.678275 4238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:00.688314 4238 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:00.688484 4238 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:00.694628 4238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c82c809d2a94ea0ba86d204b195ff21: Generated new TSK 0
I20260619 08:55:00.694815 4238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:00.696907 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 4148
2026-06-19T08:55:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:00 dist-test-slave-p0h9 krb5kdc[4247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:00 dist-test-slave-p0h9 krb5kdc[4247](info): set up 2 sockets
Jun 19 08:55:00 dist-test-slave-p0h9 krb5kdc[4247](info): commencing operation
krb5kdc: starting...
W20260619 08:55:01.725363 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:01 dist-test-slave-p0h9 krb5kdc[4247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:55:01.841528 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33877
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:01.922976 4260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:01.923251 4260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:01.923295 4260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:01.924453 4260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:01.924531 4260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:01.924556 4260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:01.924577 4260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:01.924603 4260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:01.926075 4260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:01.926362 4260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:01.926595 4260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:01.931403 4265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:01.931460 4260 server_base.cc:1034] running on GCE node
W20260619 08:55:01.931412 4266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:01.931425 4268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:01.931926 4260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:01.932278 4260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:01.933439 4260 hybrid_clock.cc:648] HybridClock initialized: now 1781859301933429 us; error 35 us; skew 500 ppm
Jun 19 08:55:01 dist-test-slave-p0h9 krb5kdc[4247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859301, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:01.935932 4260 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:55:01.936395 4260 webserver.cc:460] Webserver started at http://127.0.0.1:42537/ using document root <none> and password file <none>
I20260619 08:55:01.936638 4260 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:01.936702 4260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:01.936870 4260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:01.937920 4260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "7d73b1e24b494f2cac63724cb1d5d3e1"
format_stamp: "Formatted at 2026-06-19 08:55:01 on dist-test-slave-p0h9"
server_key: "ba4e3804edb34c1aa93a70a8a48f787e"
server_key_iv: "a190250b6f0573a10a20ca56390e405a"
server_key_version: "encryptionkey@0"
I20260619 08:55:01.938271 4260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "7d73b1e24b494f2cac63724cb1d5d3e1"
format_stamp: "Formatted at 2026-06-19 08:55:01 on dist-test-slave-p0h9"
server_key: "ba4e3804edb34c1aa93a70a8a48f787e"
server_key_iv: "a190250b6f0573a10a20ca56390e405a"
server_key_version: "encryptionkey@0"
I20260619 08:55:01.939551 4260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:55:01.940248 4275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:01.940405 4260 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:55:01.940491 4260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "7d73b1e24b494f2cac63724cb1d5d3e1"
format_stamp: "Formatted at 2026-06-19 08:55:01 on dist-test-slave-p0h9"
server_key: "ba4e3804edb34c1aa93a70a8a48f787e"
server_key_iv: "a190250b6f0573a10a20ca56390e405a"
server_key_version: "encryptionkey@0"
I20260619 08:55:01.940568 4260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:01.952706 4260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:01.955093 4260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:01.955252 4260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:01.959334 4260 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33877
I20260619 08:55:01.959370 4327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33877 every 8 connection(s)
I20260619 08:55:01.959715 4260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:01.960532 4328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:01.962774 4328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1: Bootstrap starting.
I20260619 08:55:01.963294 4328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:01.963529 4328 log.cc:824] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:01.964185 4328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1: No bootstrap required, opened a new log
I20260619 08:55:01.965778 4328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d73b1e24b494f2cac63724cb1d5d3e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33877 } }
I20260619 08:55:01.965912 4328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:01.965957 4328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d73b1e24b494f2cac63724cb1d5d3e1, State: Initialized, Role: FOLLOWER
I20260619 08:55:01.966137 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 4260
I20260619 08:55:01.966122 4328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d73b1e24b494f2cac63724cb1d5d3e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33877 } }
I20260619 08:55:01.966204 4328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:01.966240 4328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:01.966248 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:01.966292 4328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:01.966449 16192 external_mini_cluster.cc:1428] Setting key 9064122ec799663083105a828ea55254
I20260619 08:55:01.966985 4328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d73b1e24b494f2cac63724cb1d5d3e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33877 } }
I20260619 08:55:01.967144 4328 leader_election.cc:302] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d73b1e24b494f2cac63724cb1d5d3e1; no voters:
I20260619 08:55:01.967327 4328 leader_election.cc:288] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:01.967367 4333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:01.967509 4333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [term 1 LEADER]: Becoming Leader. State: Replica: 7d73b1e24b494f2cac63724cb1d5d3e1, State: Running, Role: LEADER
I20260619 08:55:01.967571 4328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:01.967702 4333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d73b1e24b494f2cac63724cb1d5d3e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33877 } }
I20260619 08:55:01.968146 4335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d73b1e24b494f2cac63724cb1d5d3e1. Latest consensus state: current_term: 1 leader_uuid: "7d73b1e24b494f2cac63724cb1d5d3e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d73b1e24b494f2cac63724cb1d5d3e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33877 } } }
I20260619 08:55:01.968232 4335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:01.968255 4334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d73b1e24b494f2cac63724cb1d5d3e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d73b1e24b494f2cac63724cb1d5d3e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33877 } } }
I20260619 08:55:01.968333 4334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:01.968786 4337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:01.969281 4337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:01.970865 4337 catalog_manager.cc:1347] Generated new cluster ID: ae1c34cfc4ae40bba3b0320f941f3ddd
I20260619 08:55:01.970918 4337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:55:01 dist-test-slave-p0h9 krb5kdc[4247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:55:01.977672 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:01.966921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42409 (local address 127.0.0.1:33877)
0619 08:55:01.967117 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:01.967127 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:01.967160 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:55:01.967278 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:01.967286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:01.967344 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:01.967493 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:01.968465 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:01.970017 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:01.971064 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:01.971495 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:01.971614 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:01.974054 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:01.974058 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:01.974058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:01.974070 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:01.976129 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:01.976490 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:01.976493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:01.976493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:01.976594 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:01.976821 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:01.976822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:01.976822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:01.977054 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:01.977239 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:01.977305 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 08:55:01.977443 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":144,"thread_start_us":99,"threads_started":1}
I20260619 08:55:01.978515 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:01.979657 4337 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:01.980325 4337 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:01.984521 4337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d73b1e24b494f2cac63724cb1d5d3e1: Generated new TSK 0
I20260619 08:55:01.984689 4337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:01.986078 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:01.979819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49361 (local address 127.0.0.1:33877)
0619 08:55:01.979857 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:01.979859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:01.980002 (+ 143us) server_negotiation.cc:407] Connection header received
0619 08:55:01.980053 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:01.980054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:01.980065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:01.980118 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:01.980685 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:01.981201 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:01.981987 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:01.982203 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:01.982282 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:01.983413 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:01.983415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:01.983415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:01.983421 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:01.984988 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:01.985335 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:01.985336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:01.985337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:01.985388 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:01.985600 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:01.985601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:01.985601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:01.985705 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:01.985803 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:01.985899 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 08:55:01.985974 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:55:01.987967 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 4260
2026-06-19T08:55:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:02 dist-test-slave-p0h9 krb5kdc[4359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:02 dist-test-slave-p0h9 krb5kdc[4359](info): set up 2 sockets
Jun 19 08:55:02 dist-test-slave-p0h9 krb5kdc[4359](info): commencing operation
krb5kdc: starting...
W20260619 08:55:03.013377 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:03 dist-test-slave-p0h9 krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:55:03.129822 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33787
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:03.209940 4372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:03.210180 4372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:03.210207 4372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:03.211315 4372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:03.211357 4372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:03.211372 4372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:03.211385 4372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:03.211396 4372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:03.212878 4372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:03.213066 4372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:03.213236 4372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:03.217945 4380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:03.217945 4377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:03.217978 4378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:03.218140 4372 server_base.cc:1034] running on GCE node
I20260619 08:55:03.218489 4372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:03.218802 4372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:03.219965 4372 hybrid_clock.cc:648] HybridClock initialized: now 1781859303219934 us; error 49 us; skew 500 ppm
Jun 19 08:55:03 dist-test-slave-p0h9 krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859303, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:03.222486 4372 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:55:03.222957 4372 webserver.cc:460] Webserver started at http://127.0.0.1:39999/ using document root <none> and password file <none>
I20260619 08:55:03.223181 4372 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:03.223255 4372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:03.223405 4372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:03.224480 4372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "7e7be1d4292649f29bb5888617dca3d6"
format_stamp: "Formatted at 2026-06-19 08:55:03 on dist-test-slave-p0h9"
server_key: "d22c02a294d37ae4965689f3411c2e6f"
server_key_iv: "d69ce1cbdd55b7b565df292354602614"
server_key_version: "encryptionkey@0"
I20260619 08:55:03.224879 4372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "7e7be1d4292649f29bb5888617dca3d6"
format_stamp: "Formatted at 2026-06-19 08:55:03 on dist-test-slave-p0h9"
server_key: "d22c02a294d37ae4965689f3411c2e6f"
server_key_iv: "d69ce1cbdd55b7b565df292354602614"
server_key_version: "encryptionkey@0"
I20260619 08:55:03.226192 4372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:55:03.226989 4387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:03.227193 4372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:55:03.227288 4372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "7e7be1d4292649f29bb5888617dca3d6"
format_stamp: "Formatted at 2026-06-19 08:55:03 on dist-test-slave-p0h9"
server_key: "d22c02a294d37ae4965689f3411c2e6f"
server_key_iv: "d69ce1cbdd55b7b565df292354602614"
server_key_version: "encryptionkey@0"
I20260619 08:55:03.227370 4372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:03.249203 4372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:03.251542 4372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:03.251729 4372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:03.256217 4372 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33787
I20260619 08:55:03.256268 4439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33787 every 8 connection(s)
I20260619 08:55:03.256652 4372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:03.257575 4440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:03.259899 4440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6: Bootstrap starting.
I20260619 08:55:03.260501 4440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:03.260856 4440 log.cc:824] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:03.261592 4440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6: No bootstrap required, opened a new log
I20260619 08:55:03.263190 4440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7be1d4292649f29bb5888617dca3d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33787 } }
I20260619 08:55:03.263340 4440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:03.263394 4440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7be1d4292649f29bb5888617dca3d6, State: Initialized, Role: FOLLOWER
I20260619 08:55:03.263557 4440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7be1d4292649f29bb5888617dca3d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33787 } }
I20260619 08:55:03.263620 4440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:03.263657 4440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:03.263711 4440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:03.264240 4440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7be1d4292649f29bb5888617dca3d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33787 } }
I20260619 08:55:03.264349 4440 leader_election.cc:302] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e7be1d4292649f29bb5888617dca3d6; no voters:
I20260619 08:55:03.264534 4440 leader_election.cc:288] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:03.264559 4443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:03.264649 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 4372
I20260619 08:55:03.264681 4443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 7e7be1d4292649f29bb5888617dca3d6, State: Running, Role: LEADER
I20260619 08:55:03.264822 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:03.264829 4440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:03.264813 4443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7be1d4292649f29bb5888617dca3d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33787 } }
I20260619 08:55:03.265018 16192 external_mini_cluster.cc:1428] Setting key f8062888bef950cebc7ca3d96b360445
I20260619 08:55:03.265126 4443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e7be1d4292649f29bb5888617dca3d6. Latest consensus state: current_term: 1 leader_uuid: "7e7be1d4292649f29bb5888617dca3d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7be1d4292649f29bb5888617dca3d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33787 } } }
I20260619 08:55:03.265187 4443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:03.265298 4444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e7be1d4292649f29bb5888617dca3d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7be1d4292649f29bb5888617dca3d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33787 } } }
I20260619 08:55:03.265378 4444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6 [sys.catalog]: This master's current role is: LEADER
W20260619 08:55:03.266729 4461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:55:03.266824 4461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:55:03.266883 4448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:03.267025 4448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:03.268476 4448 catalog_manager.cc:1347] Generated new cluster ID: b8842ca305ed43f29e4149fb6216ad74
I20260619 08:55:03.268527 4448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:03.270862 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:03.265772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40433 (local address 127.0.0.1:33787)
0619 08:55:03.265985 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:03.265996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:03.266099 (+ 103us) server_negotiation.cc:407] Connection header received
0619 08:55:03.266232 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:03.266240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:03.266300 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:03.266442 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:03.266759 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:03.267951 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:03.268965 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:03.269380 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:03.269519 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:03.270178 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:03.270181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:03.270182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:03.270381 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:03.270412 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:03.270423 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:03.270500 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:03.270578 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 08:55:03.270641 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":111,"threads_started":1}
I20260619 08:55:03.271751 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:03.279634 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:03.273152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36109 (local address 127.0.0.1:33787)
0619 08:55:03.273213 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:03.273215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:03.276817 (+ 3602us) server_negotiation.cc:407] Connection header received
0619 08:55:03.276893 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:03.276894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:03.276907 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:03.276964 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:03.277118 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:03.277665 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:03.278397 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:03.278616 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:03.278690 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:03.279356 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:03.279357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:03.279357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:03.279363 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:03.279378 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:03.279382 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:03.279437 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:03.279496 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 08:55:03.279529 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:55:03.307550 4448 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:03.308151 4448 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:03.313496 4448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e7be1d4292649f29bb5888617dca3d6: Generated new TSK 0
I20260619 08:55:03.313695 4448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:03.322958 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 4372
2026-06-19T08:55:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:03 dist-test-slave-p0h9 krb5kdc[4471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:03 dist-test-slave-p0h9 krb5kdc[4471](info): set up 2 sockets
Jun 19 08:55:03 dist-test-slave-p0h9 krb5kdc[4471](info): commencing operation
krb5kdc: starting...
W20260619 08:55:04.363114 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:04 dist-test-slave-p0h9 krb5kdc[4471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:55:04.481837 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:04.562327 4484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:04.562597 4484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:04.562637 4484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:04.563804 4484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:04.563879 4484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:04.563905 4484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:04.563927 4484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:04.563956 4484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:04.565429 4484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:04.565718 4484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:04.565949 4484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:04.570885 4489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:04.570909 4490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:04.570909 4492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:04.571141 4484 server_base.cc:1034] running on GCE node
I20260619 08:55:04.571386 4484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:04.571714 4484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:04.572916 4484 hybrid_clock.cc:648] HybridClock initialized: now 1781859304572891 us; error 34 us; skew 500 ppm
Jun 19 08:55:04 dist-test-slave-p0h9 krb5kdc[4471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859304, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:04.580078 4484 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:55:04.580621 4484 webserver.cc:460] Webserver started at http://127.0.0.1:44231/ using document root <none> and password file <none>
I20260619 08:55:04.580914 4484 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:04.580993 4484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:04.581149 4484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:04.582253 4484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "5df2869d62a84d25a94a477f1e6f0caf"
format_stamp: "Formatted at 2026-06-19 08:55:04 on dist-test-slave-p0h9"
server_key: "a8ec779c7fe51552652604bcfcd58619"
server_key_iv: "6232662e2ba90182684726fe40ba5b0b"
server_key_version: "encryptionkey@0"
I20260619 08:55:04.582645 4484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "5df2869d62a84d25a94a477f1e6f0caf"
format_stamp: "Formatted at 2026-06-19 08:55:04 on dist-test-slave-p0h9"
server_key: "a8ec779c7fe51552652604bcfcd58619"
server_key_iv: "6232662e2ba90182684726fe40ba5b0b"
server_key_version: "encryptionkey@0"
I20260619 08:55:04.584012 4484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:55:04.584775 4499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:04.584956 4484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:55:04.585057 4484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "5df2869d62a84d25a94a477f1e6f0caf"
format_stamp: "Formatted at 2026-06-19 08:55:04 on dist-test-slave-p0h9"
server_key: "a8ec779c7fe51552652604bcfcd58619"
server_key_iv: "6232662e2ba90182684726fe40ba5b0b"
server_key_version: "encryptionkey@0"
I20260619 08:55:04.585139 4484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:04.588518 4484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:04.590405 4484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:04.590600 4484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:04.594708 4484 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40503
I20260619 08:55:04.594739 4551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40503 every 8 connection(s)
I20260619 08:55:04.595077 4484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:04.595916 4552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:04.596247 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 4484
I20260619 08:55:04.596372 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:04.596577 16192 external_mini_cluster.cc:1428] Setting key 82c65db655cf3f784f0c2e96d6ffac33
I20260619 08:55:04.599089 4552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf: Bootstrap starting.
I20260619 08:55:04.599115 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:04.597169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41347 (local address 127.0.0.1:40503)
0619 08:55:04.597371 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:04.597378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:04.597403 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:55:04.597830 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:04.597836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:04.597861 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:04.597980 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:04.598650 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:04.598653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:04.598653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:04.598657 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:04.598697 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:04.598708 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:04.598764 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:04.598856 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 08:55:04.598888 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260619 08:55:04.599826 4552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:04.600147 4552 log.cc:824] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:04.600186 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:04.600979 4552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf: No bootstrap required, opened a new log
I20260619 08:55:04.602752 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:04.601750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53451 (local address 127.0.0.1:40503)
0619 08:55:04.601790 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:04.601792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:04.601859 (+ 67us) server_negotiation.cc:407] Connection header received
0619 08:55:04.601933 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:04.601933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:04.601936 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:04.601981 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:04.602517 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:04.602519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:04.602519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:04.602522 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:04.602534 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:04.602537 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:04.602577 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:04.602650 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 08:55:04.602660 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:55:04.603593 4552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5df2869d62a84d25a94a477f1e6f0caf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40503 } }
I20260619 08:55:04.603758 4552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:04.603807 4552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df2869d62a84d25a94a477f1e6f0caf, State: Initialized, Role: FOLLOWER
I20260619 08:55:04.603941 4552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5df2869d62a84d25a94a477f1e6f0caf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40503 } }
I20260619 08:55:04.604039 4552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:04.604094 4552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:04.604144 4552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:04.604691 4552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5df2869d62a84d25a94a477f1e6f0caf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40503 } }
I20260619 08:55:04.604830 4552 leader_election.cc:302] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5df2869d62a84d25a94a477f1e6f0caf; no voters:
I20260619 08:55:04.605011 4552 leader_election.cc:288] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:04.605060 4562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:04.605244 4562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [term 1 LEADER]: Becoming Leader. State: Replica: 5df2869d62a84d25a94a477f1e6f0caf, State: Running, Role: LEADER
I20260619 08:55:04.605274 4552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:04.605399 4562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5df2869d62a84d25a94a477f1e6f0caf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40503 } }
I20260619 08:55:04.605798 4564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5df2869d62a84d25a94a477f1e6f0caf. Latest consensus state: current_term: 1 leader_uuid: "5df2869d62a84d25a94a477f1e6f0caf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5df2869d62a84d25a94a477f1e6f0caf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40503 } } }
I20260619 08:55:04.605890 4564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:04.606079 4563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5df2869d62a84d25a94a477f1e6f0caf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5df2869d62a84d25a94a477f1e6f0caf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40503 } } }
I20260619 08:55:04.606139 4563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:04.606482 4570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:04.606956 4570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:04.608467 4570 catalog_manager.cc:1347] Generated new cluster ID: 4babf3cf92324f4d9b3eb9760995ddf8
I20260619 08:55:04.608529 4570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:04.618230 4570 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:04.618398 4570 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:04.620754 4570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5df2869d62a84d25a94a477f1e6f0caf: Generated new TSK 0
I20260619 08:55:04.620949 4570 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:04.624634 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 4484
2026-06-19T08:55:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:04 dist-test-slave-p0h9 krb5kdc[4583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:04 dist-test-slave-p0h9 krb5kdc[4583](info): set up 2 sockets
Jun 19 08:55:04 dist-test-slave-p0h9 krb5kdc[4583](info): commencing operation
krb5kdc: starting...
W20260619 08:55:05.661331 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:05 dist-test-slave-p0h9 krb5kdc[4583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:55:05.777927 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:05.860000 4596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:05.860292 4596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:05.860337 4596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:05.861545 4596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:05.861617 4596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:05.861642 4596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:05.861671 4596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:05.861702 4596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:05.863188 4596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:05.863492 4596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:05.863731 4596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:05.868659 4604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:05.868685 4602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:05.868709 4601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:05.868716 4596 server_base.cc:1034] running on GCE node
I20260619 08:55:05.869093 4596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:05.869370 4596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:05.870554 4596 hybrid_clock.cc:648] HybridClock initialized: now 1781859305870535 us; error 49 us; skew 500 ppm
Jun 19 08:55:05 dist-test-slave-p0h9 krb5kdc[4583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859305, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:05.873461 4596 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:55:05.874007 4596 webserver.cc:460] Webserver started at http://127.0.0.1:33555/ using document root <none> and password file <none>
I20260619 08:55:05.874192 4596 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:05.874233 4596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:05.874325 4596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:05.875497 4596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "bd207da781a24aeca652de858347aa21"
format_stamp: "Formatted at 2026-06-19 08:55:05 on dist-test-slave-p0h9"
server_key: "a6c327fdefade1f0ee527ae578adbdfb"
server_key_iv: "8fa19c799894d5f1e18108594cc6532d"
server_key_version: "encryptionkey@0"
I20260619 08:55:05.875818 4596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "bd207da781a24aeca652de858347aa21"
format_stamp: "Formatted at 2026-06-19 08:55:05 on dist-test-slave-p0h9"
server_key: "a6c327fdefade1f0ee527ae578adbdfb"
server_key_iv: "8fa19c799894d5f1e18108594cc6532d"
server_key_version: "encryptionkey@0"
I20260619 08:55:05.877187 4596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:55:05.877948 4611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:05.878121 4596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:55:05.878232 4596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "bd207da781a24aeca652de858347aa21"
format_stamp: "Formatted at 2026-06-19 08:55:05 on dist-test-slave-p0h9"
server_key: "a6c327fdefade1f0ee527ae578adbdfb"
server_key_iv: "8fa19c799894d5f1e18108594cc6532d"
server_key_version: "encryptionkey@0"
I20260619 08:55:05.878307 4596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:05.894673 4596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:05.896812 4596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:05.896982 4596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:05.901155 4596 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43551
I20260619 08:55:05.901198 4663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43551 every 8 connection(s)
I20260619 08:55:05.901598 4596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:05.902470 4664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:05.902786 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 4596
I20260619 08:55:05.902930 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:05.903146 16192 external_mini_cluster.cc:1428] Setting key 8ce90dd7c587cbdac47850cf528797d1
I20260619 08:55:05.905308 4664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21: Bootstrap starting.
I20260619 08:55:05.905901 4664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:05.906208 4664 log.cc:824] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:05.906878 4664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21: No bootstrap required, opened a new log
I20260619 08:55:05.908912 4664 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd207da781a24aeca652de858347aa21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43551 } }
I20260619 08:55:05.909075 4664 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:05.909129 4664 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd207da781a24aeca652de858347aa21, State: Initialized, Role: FOLLOWER
I20260619 08:55:05.909298 4664 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd207da781a24aeca652de858347aa21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43551 } }
I20260619 08:55:05.909399 4664 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:05.909444 4664 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:05.909497 4664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 08:55:05 dist-test-slave-p0h9 krb5kdc[4583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:55:05.910096 4664 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd207da781a24aeca652de858347aa21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43551 } }
I20260619 08:55:05.910224 4664 leader_election.cc:302] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd207da781a24aeca652de858347aa21; no voters:
I20260619 08:55:05.910432 4664 leader_election.cc:288] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:05.910466 4669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:05.910591 4669 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [term 1 LEADER]: Becoming Leader. State: Replica: bd207da781a24aeca652de858347aa21, State: Running, Role: LEADER
I20260619 08:55:05.910738 4664 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:05.910735 4669 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd207da781a24aeca652de858347aa21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43551 } }
I20260619 08:55:05.911446 4670 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd207da781a24aeca652de858347aa21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd207da781a24aeca652de858347aa21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43551 } } }
I20260619 08:55:05.911532 4670 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:05.911756 4675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:05.912248 4671 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd207da781a24aeca652de858347aa21. Latest consensus state: current_term: 1 leader_uuid: "bd207da781a24aeca652de858347aa21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd207da781a24aeca652de858347aa21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43551 } } }
I20260619 08:55:05.912361 4675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:05.912382 4671 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:05.914466 4675 catalog_manager.cc:1347] Generated new cluster ID: a7e3ebe9b4704a9a8b2908243dc5b08b
I20260619 08:55:05.914521 4675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:05.915338 4667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:05.903698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:37845 (local address 127.0.0.1:43551)
0619 08:55:05.903889 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:05.903905 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:05.903930 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:55:05.904345 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:05.904351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:05.904399 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:05.904555 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:05.905300 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:05.906362 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:05.907186 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:05.907611 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:05.907725 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:05.910440 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:05.910444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:05.910445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:05.910463 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:05.913148 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:05.913597 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:05.913600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:05.913601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:05.913704 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:05.914000 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:05.914001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:05.914002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:05.914316 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:05.914895 (+ 579us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:05.914993 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 08:55:05.915124 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260619 08:55:05.916299 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:05.925586 4675 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:05.926213 4675 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:05.930869 4667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:05.921126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:51305 (local address 127.0.0.1:43551)
0619 08:55:05.921173 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:05.921175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:05.923070 (+ 1895us) server_negotiation.cc:407] Connection header received
0619 08:55:05.923263 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:05.923265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:05.923276 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:05.923310 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:05.924306 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:05.925032 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:05.926063 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:05.926404 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:05.926473 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:05.927942 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:05.927944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:05.927944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:05.927951 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:05.929601 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:05.930030 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:05.930031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:05.930031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:05.930112 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:05.930341 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:05.930342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:05.930342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:05.930479 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:05.930602 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:05.930705 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 08:55:05.930774 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:55:05.932902 4675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd207da781a24aeca652de858347aa21: Generated new TSK 0
I20260619 08:55:05.933102 4675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:05.935475 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 4596
W20260619 08:55:05.939880 4581 connection.cc:535] client connection to 127.0.0.1:43551 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43551): Connection reset by peer (error 104)
2026-06-19T08:55:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:05 dist-test-slave-p0h9 krb5kdc[4695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:05 dist-test-slave-p0h9 krb5kdc[4695](info): set up 2 sockets
Jun 19 08:55:05 dist-test-slave-p0h9 krb5kdc[4695](info): commencing operation
krb5kdc: starting...
W20260619 08:55:06.989425 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:07 dist-test-slave-p0h9 krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:55:07.107314 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:07.188989 4708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:07.189249 4708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:07.189291 4708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:07.190598 4708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:07.190670 4708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:07.190701 4708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:07.190735 4708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:07.190763 4708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:07.192293 4708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:07.192587 4708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:07.192834 4708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:07.197747 4716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:07.197753 4714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:07.197762 4713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:07.198078 4708 server_base.cc:1034] running on GCE node
I20260619 08:55:07.198328 4708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:07.198632 4708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:07.199818 4708 hybrid_clock.cc:648] HybridClock initialized: now 1781859307199816 us; error 37 us; skew 500 ppm
Jun 19 08:55:07 dist-test-slave-p0h9 krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859307, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:07.202342 4708 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:55:07.202791 4708 webserver.cc:460] Webserver started at http://127.0.0.1:45531/ using document root <none> and password file <none>
I20260619 08:55:07.203030 4708 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:07.203102 4708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:07.203274 4708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:07.204427 4708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "ccbc7658c20240a8aef860b9a93d56b7"
format_stamp: "Formatted at 2026-06-19 08:55:07 on dist-test-slave-p0h9"
server_key: "bc9f3f14ad30592cdbf1b299f648a831"
server_key_iv: "9766590cd2a16ed7415f916d3e3d163b"
server_key_version: "encryptionkey@0"
I20260619 08:55:07.204887 4708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "ccbc7658c20240a8aef860b9a93d56b7"
format_stamp: "Formatted at 2026-06-19 08:55:07 on dist-test-slave-p0h9"
server_key: "bc9f3f14ad30592cdbf1b299f648a831"
server_key_iv: "9766590cd2a16ed7415f916d3e3d163b"
server_key_version: "encryptionkey@0"
I20260619 08:55:07.206334 4708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:55:07.207182 4723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:07.207376 4708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:55:07.207558 4708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "ccbc7658c20240a8aef860b9a93d56b7"
format_stamp: "Formatted at 2026-06-19 08:55:07 on dist-test-slave-p0h9"
server_key: "bc9f3f14ad30592cdbf1b299f648a831"
server_key_iv: "9766590cd2a16ed7415f916d3e3d163b"
server_key_version: "encryptionkey@0"
I20260619 08:55:07.207651 4708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:07.231041 4708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:07.233291 4708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:07.233471 4708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:07.237521 4708 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43147
I20260619 08:55:07.237552 4775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43147 every 8 connection(s)
I20260619 08:55:07.237926 4708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:07.238791 4776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:07.241101 4776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7: Bootstrap starting.
I20260619 08:55:07.241688 4776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:07.241982 4776 log.cc:824] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:07.242435 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 4708
I20260619 08:55:07.242583 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:07.242807 4776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7: No bootstrap required, opened a new log
I20260619 08:55:07.242821 16192 external_mini_cluster.cc:1428] Setting key 96b5153e871a7306f1db98b3dc62821b
I20260619 08:55:07.244891 4776 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbc7658c20240a8aef860b9a93d56b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } }
I20260619 08:55:07.245024 4776 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:07.245046 4776 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbc7658c20240a8aef860b9a93d56b7, State: Initialized, Role: FOLLOWER
I20260619 08:55:07.245208 4776 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbc7658c20240a8aef860b9a93d56b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } }
I20260619 08:55:07.245270 4776 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:07.245292 4776 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:07.245357 4776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:07.245918 4776 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbc7658c20240a8aef860b9a93d56b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } }
I20260619 08:55:07.246002 4776 leader_election.cc:302] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccbc7658c20240a8aef860b9a93d56b7; no voters:
I20260619 08:55:07.246244 4776 leader_election.cc:288] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:07.246279 4781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:07.246394 4781 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [term 1 LEADER]: Becoming Leader. State: Replica: ccbc7658c20240a8aef860b9a93d56b7, State: Running, Role: LEADER
I20260619 08:55:07.246527 4776 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:07.246536 4781 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbc7658c20240a8aef860b9a93d56b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } }
I20260619 08:55:07.246982 4783 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccbc7658c20240a8aef860b9a93d56b7. Latest consensus state: current_term: 1 leader_uuid: "ccbc7658c20240a8aef860b9a93d56b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbc7658c20240a8aef860b9a93d56b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } } }
I20260619 08:55:07.247074 4783 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:07.247292 4782 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccbc7658c20240a8aef860b9a93d56b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbc7658c20240a8aef860b9a93d56b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } } }
I20260619 08:55:07.247411 4782 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:07.247562 4788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:07.248173 4788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:07.249724 4780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:07.243359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:36091 (local address 127.0.0.1:43147)
0619 08:55:07.243581 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:07.243591 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:07.243626 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:55:07.244157 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:07.244164 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:07.244213 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:07.244357 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:07.244557 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:07.245951 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:07.247095 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:07.247708 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:07.247815 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:07.248748 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:07.248752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:07.248753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:07.249041 (+ 288us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:07.249091 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:07.249108 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:07.249189 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:07.249486 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 08:55:07.249513 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260619 08:55:07.250312 4788 catalog_manager.cc:1347] Generated new cluster ID: 0624bb9dbe0944d8bd64f9cd9b310dcb
I20260619 08:55:07.250385 4788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:07.250780 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:07.255780 4780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:07.252173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:59003 (local address 127.0.0.1:43147)
0619 08:55:07.252213 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:07.252215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:07.252365 (+ 150us) server_negotiation.cc:407] Connection header received
0619 08:55:07.252409 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:07.252410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:07.252421 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:07.252487 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:07.252621 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:07.253381 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:07.254413 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:07.254780 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:07.254855 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:07.255508 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:07.255510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:07.255510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:07.255514 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:07.255528 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:07.255533 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:07.255599 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:07.255694 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 08:55:07.255706 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:55:07.258391 4788 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:07.258900 4788 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:07.264335 4788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccbc7658c20240a8aef860b9a93d56b7: Generated new TSK 0
I20260619 08:55:07.264521 4788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:07.268643 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 4708
W20260619 08:55:07.273356 4693 connection.cc:535] client connection to 127.0.0.1:43147 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43147): Connection reset by peer (error 104)
2026-06-19T08:55:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:07 dist-test-slave-p0h9 krb5kdc[4807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:07 dist-test-slave-p0h9 krb5kdc[4807](info): set up 2 sockets
Jun 19 08:55:07 dist-test-slave-p0h9 krb5kdc[4807](info): commencing operation
krb5kdc: starting...
W20260619 08:55:08.305469 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:08 dist-test-slave-p0h9 krb5kdc[4807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:55:08.426828 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44867
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:08.507661 4820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:08.507915 4820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:08.507959 4820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:08.509135 4820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:08.509207 4820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:08.509231 4820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:08.509253 4820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:08.509282 4820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:08.510730 4820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:08.511019 4820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:08.511250 4820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:08.516010 4826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:08.516034 4825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:08.516216 4828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:08.516252 4820 server_base.cc:1034] running on GCE node
I20260619 08:55:08.516489 4820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:08.516835 4820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:08.518002 4820 hybrid_clock.cc:648] HybridClock initialized: now 1781859308517983 us; error 38 us; skew 500 ppm
Jun 19 08:55:08 dist-test-slave-p0h9 krb5kdc[4807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859308, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:08.520452 4820 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:55:08.520985 4820 webserver.cc:460] Webserver started at http://127.0.0.1:42995/ using document root <none> and password file <none>
I20260619 08:55:08.521190 4820 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:08.521230 4820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:08.521318 4820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:08.522352 4820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "9cb468dd5dd84d2f92fe2c160dda3f52"
format_stamp: "Formatted at 2026-06-19 08:55:08 on dist-test-slave-p0h9"
server_key: "1d45800046e59ce630c4485bbe39fe56"
server_key_iv: "4d4781b7c65c718f733d7e8ea07b191f"
server_key_version: "encryptionkey@0"
I20260619 08:55:08.522655 4820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "9cb468dd5dd84d2f92fe2c160dda3f52"
format_stamp: "Formatted at 2026-06-19 08:55:08 on dist-test-slave-p0h9"
server_key: "1d45800046e59ce630c4485bbe39fe56"
server_key_iv: "4d4781b7c65c718f733d7e8ea07b191f"
server_key_version: "encryptionkey@0"
I20260619 08:55:08.523938 4820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:55:08.524642 4835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:08.524816 4820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:08.524906 4820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "9cb468dd5dd84d2f92fe2c160dda3f52"
format_stamp: "Formatted at 2026-06-19 08:55:08 on dist-test-slave-p0h9"
server_key: "1d45800046e59ce630c4485bbe39fe56"
server_key_iv: "4d4781b7c65c718f733d7e8ea07b191f"
server_key_version: "encryptionkey@0"
I20260619 08:55:08.524960 4820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:08.528119 4820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:08.530406 4820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:08.530578 4820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:08.534668 4820 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44867
I20260619 08:55:08.534735 4887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44867 every 8 connection(s)
I20260619 08:55:08.535071 4820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:08.536134 4888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:08.539198 4888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52: Bootstrap starting.
I20260619 08:55:08.539731 4888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:08.539963 4888 log.cc:824] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:08.540653 4888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52: No bootstrap required, opened a new log
I20260619 08:55:08.541854 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 4820
I20260619 08:55:08.541997 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:08.542208 16192 external_mini_cluster.cc:1428] Setting key 376faa2a6ccfb6cc1aee62719413d47c
I20260619 08:55:08.542462 4888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb468dd5dd84d2f92fe2c160dda3f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44867 } }
I20260619 08:55:08.542622 4888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:08.542685 4888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb468dd5dd84d2f92fe2c160dda3f52, State: Initialized, Role: FOLLOWER
I20260619 08:55:08.542881 4888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb468dd5dd84d2f92fe2c160dda3f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44867 } }
I20260619 08:55:08.542989 4888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:08.543045 4888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:08.543098 4888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:08.544282 4888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb468dd5dd84d2f92fe2c160dda3f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44867 } }
I20260619 08:55:08.544394 4888 leader_election.cc:302] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cb468dd5dd84d2f92fe2c160dda3f52; no voters:
I20260619 08:55:08.544529 4892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:08.542721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:58995 (local address 127.0.0.1:44867)
0619 08:55:08.542938 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:08.542946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:08.542971 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:55:08.543147 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:08.543154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:08.543184 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:08.543317 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:08.544024 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:08.544027 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:08.544027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:08.544031 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:08.544071 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:08.544082 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:08.544144 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:08.544243 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 08:55:08.544314 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":109,"threads_started":1}
I20260619 08:55:08.544598 4893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:08.544562 4888 leader_election.cc:288] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:08.544713 4893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb468dd5dd84d2f92fe2c160dda3f52, State: Running, Role: LEADER
I20260619 08:55:08.544900 4893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb468dd5dd84d2f92fe2c160dda3f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44867 } }
I20260619 08:55:08.545038 4888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:08.545326 4894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cb468dd5dd84d2f92fe2c160dda3f52. Latest consensus state: current_term: 1 leader_uuid: "9cb468dd5dd84d2f92fe2c160dda3f52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb468dd5dd84d2f92fe2c160dda3f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44867 } } }
I20260619 08:55:08.545397 4894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:08.545351 4895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cb468dd5dd84d2f92fe2c160dda3f52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb468dd5dd84d2f92fe2c160dda3f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44867 } } }
I20260619 08:55:08.545445 4895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:08.545555 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:08.545771 4897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:08.546305 4897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:08.548383 4892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:08.547205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:43311 (local address 127.0.0.1:44867)
0619 08:55:08.547245 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:08.547247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:08.547490 (+ 243us) server_negotiation.cc:407] Connection header received
0619 08:55:08.547546 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:08.547546 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:08.547550 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:08.547597 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:08.548141 (+ 544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:08.548142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:08.548142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:08.548145 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:08.548157 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:08.548161 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:08.548225 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:08.548282 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 08:55:08.548293 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:55:08.549811 4897 catalog_manager.cc:1347] Generated new cluster ID: d1fc5f1edcb14922b5fb279a09175e1e
I20260619 08:55:08.549880 4897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:08.567572 4897 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:08.567778 4897 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:08.571611 4897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cb468dd5dd84d2f92fe2c160dda3f52: Generated new TSK 0
I20260619 08:55:08.571813 4897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:08.573333 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 4820
2026-06-19T08:55:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:08 dist-test-slave-p0h9 krb5kdc[4919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:08 dist-test-slave-p0h9 krb5kdc[4919](info): set up 2 sockets
Jun 19 08:55:08 dist-test-slave-p0h9 krb5kdc[4919](info): commencing operation
krb5kdc: starting...
W20260619 08:55:09.609477 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:09 dist-test-slave-p0h9 krb5kdc[4919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:55:09.731662 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:09.813061 4932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:09.813356 4932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:09.813395 4932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:09.814548 4932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:09.814617 4932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:09.814643 4932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:09.814674 4932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:09.814704 4932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:09.816174 4932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34527
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:09.816452 4932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:09.816679 4932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:55:09.821549 4932 server_base.cc:1034] running on GCE node
W20260619 08:55:09.821671 4940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:09.821703 4938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:09.821770 4937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:09.822115 4932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:09.822396 4932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:09.823551 4932 hybrid_clock.cc:648] HybridClock initialized: now 1781859309823553 us; error 60 us; skew 500 ppm
Jun 19 08:55:09 dist-test-slave-p0h9 krb5kdc[4919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859309, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:09.839408 4932 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:55:09.839926 4932 webserver.cc:460] Webserver started at http://127.0.0.1:34781/ using document root <none> and password file <none>
I20260619 08:55:09.840132 4932 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:09.840173 4932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:09.840267 4932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:09.841405 4932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "f17ec0c8fce44bc8b9ff4253035116e0"
format_stamp: "Formatted at 2026-06-19 08:55:09 on dist-test-slave-p0h9"
server_key: "ceaa4e85405077bc0833db74b66eee1e"
server_key_iv: "1bb714f0028be4b69abe5b87a2e0481a"
server_key_version: "encryptionkey@0"
I20260619 08:55:09.841715 4932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "f17ec0c8fce44bc8b9ff4253035116e0"
format_stamp: "Formatted at 2026-06-19 08:55:09 on dist-test-slave-p0h9"
server_key: "ceaa4e85405077bc0833db74b66eee1e"
server_key_iv: "1bb714f0028be4b69abe5b87a2e0481a"
server_key_version: "encryptionkey@0"
I20260619 08:55:09.843011 4932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:55:09.843757 4947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:09.843948 4932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:55:09.844023 4932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "f17ec0c8fce44bc8b9ff4253035116e0"
format_stamp: "Formatted at 2026-06-19 08:55:09 on dist-test-slave-p0h9"
server_key: "ceaa4e85405077bc0833db74b66eee1e"
server_key_iv: "1bb714f0028be4b69abe5b87a2e0481a"
server_key_version: "encryptionkey@0"
I20260619 08:55:09.844091 4932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:09.863065 4932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:09.865464 4932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:09.865592 4932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:09.870324 4932 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34527
I20260619 08:55:09.870358 4999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34527 every 8 connection(s)
I20260619 08:55:09.870762 4932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:09.871683 5000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:09.874140 5000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0: Bootstrap starting.
I20260619 08:55:09.874688 5000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:09.874922 5000 log.cc:824] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:09.875633 5000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0: No bootstrap required, opened a new log
I20260619 08:55:09.876899 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 4932
I20260619 08:55:09.877035 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:09.877236 16192 external_mini_cluster.cc:1428] Setting key e48064af6a7a5d962219f15e9c44c434
I20260619 08:55:09.877758 5000 raft_consensus.cc:348] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17ec0c8fce44bc8b9ff4253035116e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } }
I20260619 08:55:09.877982 5000 raft_consensus.cc:374] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:09.878041 5000 raft_consensus.cc:729] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f17ec0c8fce44bc8b9ff4253035116e0, State: Initialized, Role: FOLLOWER
I20260619 08:55:09.878185 5000 consensus_queue.cc:260] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17ec0c8fce44bc8b9ff4253035116e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } }
I20260619 08:55:09.878264 5000 raft_consensus.cc:388] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:09.878306 5000 raft_consensus.cc:482] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:09.878367 5000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:09.879559 5000 raft_consensus.cc:504] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17ec0c8fce44bc8b9ff4253035116e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } }
I20260619 08:55:09.879655 5000 leader_election.cc:302] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f17ec0c8fce44bc8b9ff4253035116e0; no voters:
I20260619 08:55:09.879850 5000 leader_election.cc:288] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:09.879874 5005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:09.880226 5005 raft_consensus.cc:686] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [term 1 LEADER]: Becoming Leader. State: Replica: f17ec0c8fce44bc8b9ff4253035116e0, State: Running, Role: LEADER
I20260619 08:55:09.880326 5000 sys_catalog.cc:564] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:09.880385 5005 consensus_queue.cc:237] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17ec0c8fce44bc8b9ff4253035116e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } }
I20260619 08:55:09.881930 5006 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f17ec0c8fce44bc8b9ff4253035116e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17ec0c8fce44bc8b9ff4253035116e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } } }
I20260619 08:55:09.882054 5007 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f17ec0c8fce44bc8b9ff4253035116e0. Latest consensus state: current_term: 1 leader_uuid: "f17ec0c8fce44bc8b9ff4253035116e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17ec0c8fce44bc8b9ff4253035116e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } } }
I20260619 08:55:09.883106 5006 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:09.883114 5007 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0 [sys.catalog]: This master's current role is: LEADER
W20260619 08:55:09.882306 5020 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:55:09.883234 5020 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:55:09.884471 5022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:09.884706 5022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:55:09 dist-test-slave-p0h9 krb5kdc[4919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:55:09.886677 5022 catalog_manager.cc:1347] Generated new cluster ID: f5aa3629924d4767bdba67dec05b3527
I20260619 08:55:09.886729 5022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:09.889572 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:09.877794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:36091 (local address 127.0.0.1:34527)
0619 08:55:09.878008 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:09.878018 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:09.878057 (+ 39us) server_negotiation.cc:407] Connection header received
0619 08:55:09.878200 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:09.878208 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:09.878285 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:09.878458 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:09.879288 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:09.880541 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:09.881686 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:09.882353 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:09.882464 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:09.885704 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:09.885709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:09.885710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:09.885723 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:09.887792 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:09.888244 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:09.888248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:09.888248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:09.888368 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:09.888646 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:09.888647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:09.888648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:09.888977 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:09.889177 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:09.889276 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 08:55:09.889402 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260619 08:55:09.890563 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:09.898473 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:09.891994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:45447 (local address 127.0.0.1:34527)
0619 08:55:09.892032 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:09.892034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:09.892294 (+ 260us) server_negotiation.cc:407] Connection header received
0619 08:55:09.892371 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:09.892372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:09.892383 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:09.892438 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:09.893039 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:09.893541 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:09.894257 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:09.894501 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:09.894577 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:09.895776 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:09.895777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:09.895778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:09.895785 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:09.897272 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:09.897648 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:09.897649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:09.897650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:09.897709 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:09.897946 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:09.897947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:09.897947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:09.898069 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:09.898180 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:09.898307 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 08:55:09.898372 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:55:09.914127 5022 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:09.914871 5022 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:09.919824 5022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f17ec0c8fce44bc8b9ff4253035116e0: Generated new TSK 0
I20260619 08:55:09.920035 5022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:09.926429 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 4932
W20260619 08:55:09.931027 4917 connection.cc:535] client connection to 127.0.0.1:34527 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34527): Connection reset by peer (error 104)
2026-06-19T08:55:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:09 dist-test-slave-p0h9 krb5kdc[5031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:09 dist-test-slave-p0h9 krb5kdc[5031](info): set up 2 sockets
Jun 19 08:55:09 dist-test-slave-p0h9 krb5kdc[5031](info): commencing operation
krb5kdc: starting...
W20260619 08:55:10.973450 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:11 dist-test-slave-p0h9 krb5kdc[5031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:55:11.089123 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40765
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:11.169262 5044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:11.169478 5044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:11.169507 5044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:11.170656 5044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:11.170699 5044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:11.170714 5044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:11.170727 5044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:11.170739 5044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:11.172223 5044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40765
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:11.172473 5044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:11.172708 5044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:11.177647 5049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:11.177764 5044 server_base.cc:1034] running on GCE node
W20260619 08:55:11.177634 5050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:11.177618 5052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:11.178103 5044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:11.178438 5044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:11.179608 5044 hybrid_clock.cc:648] HybridClock initialized: now 1781859311179592 us; error 45 us; skew 500 ppm
Jun 19 08:55:11 dist-test-slave-p0h9 krb5kdc[5031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859311, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:11.182129 5044 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:55:11.182613 5044 webserver.cc:460] Webserver started at http://127.0.0.1:43735/ using document root <none> and password file <none>
I20260619 08:55:11.182855 5044 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:11.182924 5044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:11.183059 5044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:11.184154 5044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "a4be257dbf3e4f7babbd58a5d4748e12"
format_stamp: "Formatted at 2026-06-19 08:55:11 on dist-test-slave-p0h9"
server_key: "edb40af3c14f9e4fca7fb65fcfd5bde4"
server_key_iv: "5fb9d32b6d4391c126e1e92d47ccb4e1"
server_key_version: "encryptionkey@0"
I20260619 08:55:11.184521 5044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "a4be257dbf3e4f7babbd58a5d4748e12"
format_stamp: "Formatted at 2026-06-19 08:55:11 on dist-test-slave-p0h9"
server_key: "edb40af3c14f9e4fca7fb65fcfd5bde4"
server_key_iv: "5fb9d32b6d4391c126e1e92d47ccb4e1"
server_key_version: "encryptionkey@0"
I20260619 08:55:11.185892 5044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:55:11.186645 5059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:11.186818 5044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:55:11.186905 5044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "a4be257dbf3e4f7babbd58a5d4748e12"
format_stamp: "Formatted at 2026-06-19 08:55:11 on dist-test-slave-p0h9"
server_key: "edb40af3c14f9e4fca7fb65fcfd5bde4"
server_key_iv: "5fb9d32b6d4391c126e1e92d47ccb4e1"
server_key_version: "encryptionkey@0"
I20260619 08:55:11.187022 5044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:11.210615 5044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:11.212473 5044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:11.212647 5044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:11.216632 5044 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40765
I20260619 08:55:11.216665 5111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40765 every 8 connection(s)
I20260619 08:55:11.217017 5044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:11.217818 5112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:11.220073 5112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12: Bootstrap starting.
I20260619 08:55:11.220672 5112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:11.220991 5112 log.cc:824] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:11.221693 5112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12: No bootstrap required, opened a new log
I20260619 08:55:11.223312 5112 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4be257dbf3e4f7babbd58a5d4748e12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40765 } }
I20260619 08:55:11.223471 5112 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:11.223519 5112 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4be257dbf3e4f7babbd58a5d4748e12, State: Initialized, Role: FOLLOWER
I20260619 08:55:11.223690 5112 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4be257dbf3e4f7babbd58a5d4748e12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40765 } }
I20260619 08:55:11.223794 5112 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:11.223842 5112 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:11.223858 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 5044
I20260619 08:55:11.223891 5112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:11.224020 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:11.224217 16192 external_mini_cluster.cc:1428] Setting key c79e20d9eb65b465e0559c75e5ff97ce
I20260619 08:55:11.224630 5112 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4be257dbf3e4f7babbd58a5d4748e12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40765 } }
I20260619 08:55:11.224772 5112 leader_election.cc:302] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4be257dbf3e4f7babbd58a5d4748e12; no voters:
I20260619 08:55:11.224948 5112 leader_election.cc:288] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:11.224995 5117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:11.225147 5117 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [term 1 LEADER]: Becoming Leader. State: Replica: a4be257dbf3e4f7babbd58a5d4748e12, State: Running, Role: LEADER
I20260619 08:55:11.225240 5112 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:11.225313 5117 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4be257dbf3e4f7babbd58a5d4748e12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40765 } }
I20260619 08:55:11.225715 5117 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4be257dbf3e4f7babbd58a5d4748e12. Latest consensus state: current_term: 1 leader_uuid: "a4be257dbf3e4f7babbd58a5d4748e12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4be257dbf3e4f7babbd58a5d4748e12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40765 } } }
I20260619 08:55:11.225841 5117 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:11.225883 5119 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4be257dbf3e4f7babbd58a5d4748e12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4be257dbf3e4f7babbd58a5d4748e12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40765 } } }
I20260619 08:55:11.225986 5119 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:11.226079 5126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:11.226575 5126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:11.228065 5126 catalog_manager.cc:1347] Generated new cluster ID: a85d9679c3c14a4f8c4ffb5d63740b0d
I20260619 08:55:11.228114 5126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:11.230321 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:11.224740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:34557 (local address 127.0.0.1:40765)
0619 08:55:11.224969 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:11.224979 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:11.225016 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:55:11.225138 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:11.225146 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:11.225208 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:11.225622 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:11.225729 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:11.227125 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:11.228194 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:11.228771 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:11.228878 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:11.229643 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:11.229647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:11.229647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:11.229851 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:11.229890 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:11.229905 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:11.229987 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:11.230084 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 08:55:11.230118 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260619 08:55:11.232339 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:11.238682 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:11.233717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:39735 (local address 127.0.0.1:40765)
0619 08:55:11.233747 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:11.233748 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:11.233892 (+ 144us) server_negotiation.cc:407] Connection header received
0619 08:55:11.233919 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:11.233920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:11.233930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:11.233999 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:11.234136 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:11.235261 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:11.236247 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:11.236630 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:11.236706 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:11.238424 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:11.238426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:11.238426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:11.238431 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:11.238446 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:11.238451 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:11.238519 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:11.238597 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 08:55:11.238609 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 08:55:11.248032 5126 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:11.248814 5126 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:11.258989 5126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4be257dbf3e4f7babbd58a5d4748e12: Generated new TSK 0
I20260619 08:55:11.259203 5126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:11.264537 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 5044
W20260619 08:55:11.269048 5029 connection.cc:535] client connection to 127.0.0.1:40765 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40765): Connection reset by peer (error 104)
2026-06-19T08:55:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:11 dist-test-slave-p0h9 krb5kdc[5143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:11 dist-test-slave-p0h9 krb5kdc[5143](info): set up 2 sockets
Jun 19 08:55:11 dist-test-slave-p0h9 krb5kdc[5143](info): commencing operation
krb5kdc: starting...
W20260619 08:55:12.305541 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:12 dist-test-slave-p0h9 krb5kdc[5143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:55:12.429639 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:12.512209 5156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:12.512420 5156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:12.512445 5156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:12.513664 5156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:12.513712 5156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:12.513729 5156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:12.513743 5156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:12.513757 5156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:12.515273 5156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38497
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:12.515534 5156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:12.515798 5156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:12.520833 5161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:12.520844 5164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:12.520845 5162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:12.520876 5156 server_base.cc:1034] running on GCE node
I20260619 08:55:12.521304 5156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:12.521648 5156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:12.522809 5156 hybrid_clock.cc:648] HybridClock initialized: now 1781859312522795 us; error 44 us; skew 500 ppm
Jun 19 08:55:12 dist-test-slave-p0h9 krb5kdc[5143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859312, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:12.525331 5156 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:55:12.525806 5156 webserver.cc:460] Webserver started at http://127.0.0.1:41521/ using document root <none> and password file <none>
I20260619 08:55:12.526031 5156 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:12.526124 5156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:12.526260 5156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:12.527638 5156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "cce34627e384466a881195c2bca620a9"
format_stamp: "Formatted at 2026-06-19 08:55:12 on dist-test-slave-p0h9"
server_key: "27884d50c6567b0c942dda16f96eaed2"
server_key_iv: "f4a1e431881b74247fc61678a3301db7"
server_key_version: "encryptionkey@0"
I20260619 08:55:12.528007 5156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "cce34627e384466a881195c2bca620a9"
format_stamp: "Formatted at 2026-06-19 08:55:12 on dist-test-slave-p0h9"
server_key: "27884d50c6567b0c942dda16f96eaed2"
server_key_iv: "f4a1e431881b74247fc61678a3301db7"
server_key_version: "encryptionkey@0"
I20260619 08:55:12.529387 5156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:55:12.530140 5171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:12.530316 5156 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:55:12.530418 5156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "cce34627e384466a881195c2bca620a9"
format_stamp: "Formatted at 2026-06-19 08:55:12 on dist-test-slave-p0h9"
server_key: "27884d50c6567b0c942dda16f96eaed2"
server_key_iv: "f4a1e431881b74247fc61678a3301db7"
server_key_version: "encryptionkey@0"
I20260619 08:55:12.530498 5156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:12.533763 5156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:12.535571 5156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:12.535737 5156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:12.539755 5156 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38497
I20260619 08:55:12.539803 5223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38497 every 8 connection(s)
I20260619 08:55:12.540146 5156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:12.541044 5224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:12.543210 5224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9: Bootstrap starting.
I20260619 08:55:12.543747 5224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:12.543975 5224 log.cc:824] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:12.544420 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 5156
I20260619 08:55:12.544554 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:12.544739 5224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9: No bootstrap required, opened a new log
I20260619 08:55:12.544783 16192 external_mini_cluster.cc:1428] Setting key 0da2677aec7c5126be07f03cd34484f8
I20260619 08:55:12.546818 5224 raft_consensus.cc:348] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce34627e384466a881195c2bca620a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38497 } }
I20260619 08:55:12.546945 5224 raft_consensus.cc:374] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:12.546967 5224 raft_consensus.cc:729] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce34627e384466a881195c2bca620a9, State: Initialized, Role: FOLLOWER
I20260619 08:55:12.547106 5224 consensus_queue.cc:260] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce34627e384466a881195c2bca620a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38497 } }
I20260619 08:55:12.547220 5224 raft_consensus.cc:388] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:12.547271 5224 raft_consensus.cc:482] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:12.547325 5224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:12.547480 5228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:12.545313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:58899 (local address 127.0.0.1:38497)
0619 08:55:12.545505 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:12.545515 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:12.545550 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:55:12.545986 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:12.545992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:12.546017 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:12.546189 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:12.546885 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:12.546888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:12.546889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:12.546893 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:12.546927 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:12.546937 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:12.547022 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:12.547192 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 08:55:12.547248 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260619 08:55:12.548380 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:12.548416 5224 raft_consensus.cc:504] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce34627e384466a881195c2bca620a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38497 } }
I20260619 08:55:12.548538 5224 leader_election.cc:302] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cce34627e384466a881195c2bca620a9; no voters:
I20260619 08:55:12.548765 5224 leader_election.cc:288] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:12.548805 5229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:12.548933 5229 raft_consensus.cc:686] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [term 1 LEADER]: Becoming Leader. State: Replica: cce34627e384466a881195c2bca620a9, State: Running, Role: LEADER
I20260619 08:55:12.549011 5224 sys_catalog.cc:564] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:12.549011 5229 consensus_queue.cc:237] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce34627e384466a881195c2bca620a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38497 } }
I20260619 08:55:12.549364 5229 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cce34627e384466a881195c2bca620a9. Latest consensus state: current_term: 1 leader_uuid: "cce34627e384466a881195c2bca620a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce34627e384466a881195c2bca620a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38497 } } }
I20260619 08:55:12.549456 5229 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:12.549372 5231 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cce34627e384466a881195c2bca620a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce34627e384466a881195c2bca620a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38497 } } }
I20260619 08:55:12.549647 5231 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:12.549844 5237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:12.550474 5237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:12.553890 5228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:12.550019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:51839 (local address 127.0.0.1:38497)
0619 08:55:12.550053 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:12.550055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:12.552975 (+ 2920us) server_negotiation.cc:407] Connection header received
0619 08:55:12.553018 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:12.553019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:12.553023 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:12.553085 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:12.553692 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:12.553693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:12.553694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:12.553698 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:12.553710 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:12.553714 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:12.553759 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:12.553813 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 08:55:12.553827 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:55:12.557785 5237 catalog_manager.cc:1347] Generated new cluster ID: bf2088df37e84401bec562ea74d6e3b6
I20260619 08:55:12.557842 5237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:12.588491 5237 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:12.588702 5237 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:12.592530 5237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cce34627e384466a881195c2bca620a9: Generated new TSK 0
I20260619 08:55:12.592778 5237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:12.602300 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 5156
2026-06-19T08:55:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:12 dist-test-slave-p0h9 krb5kdc[5255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:12 dist-test-slave-p0h9 krb5kdc[5255](info): set up 2 sockets
Jun 19 08:55:12 dist-test-slave-p0h9 krb5kdc[5255](info): commencing operation
krb5kdc: starting...
W20260619 08:55:13.633283 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:13 dist-test-slave-p0h9 krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:55:13.749524 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:42291
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:42291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:13.829483 5268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:13.829732 5268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:13.829769 5268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:13.830917 5268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:13.830991 5268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:13.831015 5268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:13.831036 5268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:13.831066 5268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:13.832594 5268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:42291
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:42291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:13.832880 5268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:13.833123 5268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:13.838070 5273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:13.838146 5274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:13.838130 5276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:13.838171 5268 server_base.cc:1034] running on GCE node
I20260619 08:55:13.838515 5268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:13.838744 5268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:13.839908 5268 hybrid_clock.cc:648] HybridClock initialized: now 1781859313839895 us; error 29 us; skew 500 ppm
Jun 19 08:55:13 dist-test-slave-p0h9 krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859313, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:13.842397 5268 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:55:13.842806 5268 webserver.cc:460] Webserver started at http://127.15.208.62:45679/ using document root <none> and password file <none>
I20260619 08:55:13.843039 5268 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:13.843091 5268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:13.843268 5268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:13.844303 5268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "c3f8bdc6c4e34c26b5157394b461af5d"
format_stamp: "Formatted at 2026-06-19 08:55:13 on dist-test-slave-p0h9"
server_key: "18fad59798a3b35beffaa49dd9514cc2"
server_key_iv: "0f018d1f762a70ffb717bb44096feeae"
server_key_version: "encryptionkey@0"
I20260619 08:55:13.844657 5268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "c3f8bdc6c4e34c26b5157394b461af5d"
format_stamp: "Formatted at 2026-06-19 08:55:13 on dist-test-slave-p0h9"
server_key: "18fad59798a3b35beffaa49dd9514cc2"
server_key_iv: "0f018d1f762a70ffb717bb44096feeae"
server_key_version: "encryptionkey@0"
I20260619 08:55:13.845983 5268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:55:13.846684 5283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:13.846869 5268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:55:13.846962 5268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "c3f8bdc6c4e34c26b5157394b461af5d"
format_stamp: "Formatted at 2026-06-19 08:55:13 on dist-test-slave-p0h9"
server_key: "18fad59798a3b35beffaa49dd9514cc2"
server_key_iv: "0f018d1f762a70ffb717bb44096feeae"
server_key_version: "encryptionkey@0"
I20260619 08:55:13.847033 5268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:13.863090 5268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:13.865051 5268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:13.865214 5268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:13.869153 5268 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:42291
I20260619 08:55:13.869204 5335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:42291 every 8 connection(s)
I20260619 08:55:13.869567 5268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:13.870398 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:13.872807 5336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d: Bootstrap starting.
I20260619 08:55:13.873394 5336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:13.873703 5336 log.cc:824] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:13.874222 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 5268
I20260619 08:55:13.874394 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:13.874490 5336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d: No bootstrap required, opened a new log
I20260619 08:55:13.874663 16192 external_mini_cluster.cc:1428] Setting key 32d0ffbdb2899971c5d08eb7f37b66e8
I20260619 08:55:13.876376 5336 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f8bdc6c4e34c26b5157394b461af5d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42291 } }
I20260619 08:55:13.876538 5336 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:13.876622 5336 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f8bdc6c4e34c26b5157394b461af5d, State: Initialized, Role: FOLLOWER
I20260619 08:55:13.876808 5336 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f8bdc6c4e34c26b5157394b461af5d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42291 } }
I20260619 08:55:13.876888 5336 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:13.876911 5336 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:13.876963 5336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:13.877830 5336 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f8bdc6c4e34c26b5157394b461af5d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42291 } }
I20260619 08:55:13.877950 5336 leader_election.cc:302] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3f8bdc6c4e34c26b5157394b461af5d; no voters:
I20260619 08:55:13.878122 5336 leader_election.cc:288] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:13.878149 5341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:13.878264 5341 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [term 1 LEADER]: Becoming Leader. State: Replica: c3f8bdc6c4e34c26b5157394b461af5d, State: Running, Role: LEADER
I20260619 08:55:13.878381 5336 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:13.878401 5341 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f8bdc6c4e34c26b5157394b461af5d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42291 } }
I20260619 08:55:13.878800 5341 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3f8bdc6c4e34c26b5157394b461af5d. Latest consensus state: current_term: 1 leader_uuid: "c3f8bdc6c4e34c26b5157394b461af5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f8bdc6c4e34c26b5157394b461af5d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42291 } } }
I20260619 08:55:13.878809 5342 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3f8bdc6c4e34c26b5157394b461af5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f8bdc6c4e34c26b5157394b461af5d" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 42291 } } }
I20260619 08:55:13.879027 5342 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:13.879235 5341 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:13.879755 5350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:13.880405 5350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:13.882210 5350 catalog_manager.cc:1347] Generated new cluster ID: 06416e0fd8444d3f9eae71603c66c602
I20260619 08:55:13.882266 5350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:55:13 dist-test-slave-p0h9 krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:55:13.886598 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:13.875229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59093 (local address 127.15.208.62:42291)
0619 08:55:13.875455 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:13.875464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:13.875488 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:55:13.876361 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:13.876368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:13.876417 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:13.876567 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:13.877391 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:13.878505 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:13.879644 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:13.880262 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:13.880389 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:13.883067 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:13.883071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:13.883072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:13.883085 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:13.884849 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:13.885261 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:13.885269 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:13.885269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:13.885377 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:13.885629 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:13.885630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:13.885630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:13.885903 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:13.886078 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:13.886329 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 08:55:13.886431 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260619 08:55:13.887665 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:13.895267 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:13.889120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53893 (local address 127.15.208.62:42291)
0619 08:55:13.889175 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:13.889177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:13.889185 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:55:13.889229 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:13.889230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:13.889239 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:13.889287 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:13.889788 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:13.890273 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:13.891016 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:13.891251 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:13.891339 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:13.892507 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:13.892509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:13.892510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:13.892517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:13.894028 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:13.894404 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:13.894405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:13.894405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:13.894462 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:13.894684 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:13.894685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:13.894686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:13.894795 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:13.894917 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:13.895036 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 08:55:13.895099 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:55:13.913410 5350 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:13.914170 5350 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:13.919271 5350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3f8bdc6c4e34c26b5157394b461af5d: Generated new TSK 0
I20260619 08:55:13.919458 5350 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:13.925551 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 5268
W20260619 08:55:13.930068 5253 connection.cc:535] client connection to 127.15.208.62:42291 recv error: Network error: failed to read from TLS socket (remote: 127.15.208.62:42291): Connection reset by peer (error 104)
2026-06-19T08:55:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:13 dist-test-slave-p0h9 krb5kdc[5367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:13 dist-test-slave-p0h9 krb5kdc[5367](info): set up 2 sockets
Jun 19 08:55:13 dist-test-slave-p0h9 krb5kdc[5367](info): commencing operation
krb5kdc: starting...
W20260619 08:55:14.961512 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:15 dist-test-slave-p0h9 krb5kdc[5367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:55:15.079780 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:37145
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:37145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:15.162381 5380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:15.162621 5380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:15.162647 5380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:15.163751 5380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:15.163796 5380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:15.163813 5380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:15.163827 5380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:15.163841 5380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:15.165385 5380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:37145
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:37145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:15.165704 5380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:15.165954 5380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:15.171057 5388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:15.171134 5386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:15.171057 5385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:15.171300 5380 server_base.cc:1034] running on GCE node
I20260619 08:55:15.171567 5380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:15.171815 5380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:15.172978 5380 hybrid_clock.cc:648] HybridClock initialized: now 1781859315172963 us; error 47 us; skew 500 ppm
Jun 19 08:55:15 dist-test-slave-p0h9 krb5kdc[5367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859315, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:15.182551 5380 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:55:15.183074 5380 webserver.cc:460] Webserver started at http://127.15.208.62:40097/ using document root <none> and password file <none>
I20260619 08:55:15.183359 5380 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:15.183439 5380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:15.183606 5380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:15.184892 5380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "abff5f226d264d6d92e16384f0905722"
format_stamp: "Formatted at 2026-06-19 08:55:15 on dist-test-slave-p0h9"
server_key: "3a03ffbc05b9c8975d22e4ddca285fdd"
server_key_iv: "6d53f3e01f7a8f213f35025bcbfa0284"
server_key_version: "encryptionkey@0"
I20260619 08:55:15.185315 5380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "abff5f226d264d6d92e16384f0905722"
format_stamp: "Formatted at 2026-06-19 08:55:15 on dist-test-slave-p0h9"
server_key: "3a03ffbc05b9c8975d22e4ddca285fdd"
server_key_iv: "6d53f3e01f7a8f213f35025bcbfa0284"
server_key_version: "encryptionkey@0"
I20260619 08:55:15.186772 5380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:55:15.187566 5395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:15.187749 5380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:55:15.187867 5380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "abff5f226d264d6d92e16384f0905722"
format_stamp: "Formatted at 2026-06-19 08:55:15 on dist-test-slave-p0h9"
server_key: "3a03ffbc05b9c8975d22e4ddca285fdd"
server_key_iv: "6d53f3e01f7a8f213f35025bcbfa0284"
server_key_version: "encryptionkey@0"
I20260619 08:55:15.187951 5380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:15.217746 5380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:15.219688 5380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:15.219862 5380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:15.224051 5380 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:37145
I20260619 08:55:15.224092 5447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:37145 every 8 connection(s)
I20260619 08:55:15.224484 5380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:15.224938 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 5380
I20260619 08:55:15.225090 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:15.225306 16192 external_mini_cluster.cc:1428] Setting key 1029d5962f93e2bd7708cef7e00275f7
I20260619 08:55:15.225613 5448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:15.228377 5448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722: Bootstrap starting.
I20260619 08:55:15.229033 5448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:15.229326 5448 log.cc:824] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:15.230075 5448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722: No bootstrap required, opened a new log
I20260619 08:55:15.231096 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:15.225839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56257 (local address 127.15.208.62:37145)
0619 08:55:15.226047 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:15.226057 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:15.226090 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:55:15.226746 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:15.226753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:15.226803 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:15.226949 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:15.227189 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:15.228083 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:15.229013 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:15.229517 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:15.229655 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:15.230454 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:15.230458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:15.230458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:15.230631 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:15.230662 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:15.230673 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:15.230779 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:15.230868 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 08:55:15.230897 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260619 08:55:15.231781 5448 raft_consensus.cc:348] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abff5f226d264d6d92e16384f0905722" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37145 } }
I20260619 08:55:15.231964 5448 raft_consensus.cc:374] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:15.232023 5448 raft_consensus.cc:729] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abff5f226d264d6d92e16384f0905722, State: Initialized, Role: FOLLOWER
I20260619 08:55:15.232151 5448 consensus_queue.cc:260] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abff5f226d264d6d92e16384f0905722" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37145 } }
I20260619 08:55:15.232256 5448 raft_consensus.cc:388] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:15.232300 5448 raft_consensus.cc:482] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:15.232350 5448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:15.232519 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:15.233011 5448 raft_consensus.cc:504] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abff5f226d264d6d92e16384f0905722" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37145 } }
I20260619 08:55:15.233134 5448 leader_election.cc:302] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abff5f226d264d6d92e16384f0905722; no voters:
I20260619 08:55:15.233330 5448 leader_election.cc:288] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:15.233363 5453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:15.233480 5453 raft_consensus.cc:686] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [term 1 LEADER]: Becoming Leader. State: Replica: abff5f226d264d6d92e16384f0905722, State: Running, Role: LEADER
I20260619 08:55:15.233590 5448 sys_catalog.cc:564] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:15.233636 5453 consensus_queue.cc:237] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abff5f226d264d6d92e16384f0905722" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37145 } }
I20260619 08:55:15.234027 5453 sys_catalog.cc:455] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abff5f226d264d6d92e16384f0905722. Latest consensus state: current_term: 1 leader_uuid: "abff5f226d264d6d92e16384f0905722" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abff5f226d264d6d92e16384f0905722" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37145 } } }
I20260619 08:55:15.234061 5454 sys_catalog.cc:455] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abff5f226d264d6d92e16384f0905722" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abff5f226d264d6d92e16384f0905722" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 37145 } } }
I20260619 08:55:15.234180 5453 sys_catalog.cc:458] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:15.234236 5454 sys_catalog.cc:458] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:15.234782 5466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:15.235417 5466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:15.237105 5466 catalog_manager.cc:1347] Generated new cluster ID: e5632ed1169d4f2d9e3a9449597ba5a3
I20260619 08:55:15.237178 5466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:15.242578 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:15.235249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33827 (local address 127.15.208.62:37145)
0619 08:55:15.235291 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:15.235293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:15.235506 (+ 213us) server_negotiation.cc:407] Connection header received
0619 08:55:15.235579 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:15.235580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:15.235589 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:15.235634 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:15.235790 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:15.236383 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:15.241303 (+ 4920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:15.241517 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:15.241604 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:15.242311 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:15.242312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:15.242312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:15.242317 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:15.242332 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:15.242336 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:15.242406 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:15.242469 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 08:55:15.242479 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:55:15.243808 5466 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:15.244464 5466 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:15.247651 5466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abff5f226d264d6d92e16384f0905722: Generated new TSK 0
I20260619 08:55:15.247861 5466 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:15.250945 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 5380
W20260619 08:55:15.255445 5365 connection.cc:535] client connection to 127.15.208.62:37145 recv error: Network error: failed to read from TLS socket (remote: 127.15.208.62:37145): Connection reset by peer (error 104)
2026-06-19T08:55:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:15 dist-test-slave-p0h9 krb5kdc[5479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:15 dist-test-slave-p0h9 krb5kdc[5479](info): set up 2 sockets
Jun 19 08:55:15 dist-test-slave-p0h9 krb5kdc[5479](info): commencing operation
krb5kdc: starting...
W20260619 08:55:16.281561 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:16 dist-test-slave-p0h9 krb5kdc[5479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:55:16.399804 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:38637
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:38637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:16.479579 5492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:16.479792 5492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:16.479815 5492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:16.480922 5492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:16.480959 5492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:16.480973 5492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:16.480985 5492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:16.480997 5492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:16.482372 5492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:38637
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:38637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:16.482560 5492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:16.482734 5492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:16.487402 5498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:16.487419 5500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:16.487408 5497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:16.487804 5492 server_base.cc:1034] running on GCE node
I20260619 08:55:16.488075 5492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:16.488364 5492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:16.489524 5492 hybrid_clock.cc:648] HybridClock initialized: now 1781859316489514 us; error 42 us; skew 500 ppm
Jun 19 08:55:16 dist-test-slave-p0h9 krb5kdc[5479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859316, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:16.491895 5492 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:55:16.492339 5492 webserver.cc:460] Webserver started at http://127.15.208.62:34865/ using document root <none> and password file <none>
I20260619 08:55:16.492590 5492 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:16.492666 5492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:16.492838 5492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:16.493839 5492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "beb848918df54ffeb1bffb4bb986c3f5"
format_stamp: "Formatted at 2026-06-19 08:55:16 on dist-test-slave-p0h9"
server_key: "b4ce5c72ee940944f8139d72f4012934"
server_key_iv: "fd92b1342229233a0f1d3efb6d47cc64"
server_key_version: "encryptionkey@0"
I20260619 08:55:16.494228 5492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "beb848918df54ffeb1bffb4bb986c3f5"
format_stamp: "Formatted at 2026-06-19 08:55:16 on dist-test-slave-p0h9"
server_key: "b4ce5c72ee940944f8139d72f4012934"
server_key_iv: "fd92b1342229233a0f1d3efb6d47cc64"
server_key_version: "encryptionkey@0"
I20260619 08:55:16.495558 5492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:55:16.496315 5507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:16.496564 5492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:55:16.496683 5492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "beb848918df54ffeb1bffb4bb986c3f5"
format_stamp: "Formatted at 2026-06-19 08:55:16 on dist-test-slave-p0h9"
server_key: "b4ce5c72ee940944f8139d72f4012934"
server_key_iv: "fd92b1342229233a0f1d3efb6d47cc64"
server_key_version: "encryptionkey@0"
I20260619 08:55:16.496788 5492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:16.500088 5492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:16.502534 5492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:16.502697 5492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:16.506942 5492 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:38637
I20260619 08:55:16.506975 5559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:38637 every 8 connection(s)
I20260619 08:55:16.507308 5492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:16.508428 5560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:16.510673 5560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5: Bootstrap starting.
I20260619 08:55:16.511188 5560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:16.511415 5560 log.cc:824] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:16.512038 5560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5: No bootstrap required, opened a new log
I20260619 08:55:16.513698 5560 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb848918df54ffeb1bffb4bb986c3f5" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38637 } }
I20260619 08:55:16.513831 5560 raft_consensus.cc:374] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:16.513854 5560 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb848918df54ffeb1bffb4bb986c3f5, State: Initialized, Role: FOLLOWER
I20260619 08:55:16.513952 5560 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb848918df54ffeb1bffb4bb986c3f5" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38637 } }
I20260619 08:55:16.514003 5560 raft_consensus.cc:388] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:16.514024 5560 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:16.514094 5560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:16.514581 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 5492
I20260619 08:55:16.514715 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:16.514937 16192 external_mini_cluster.cc:1428] Setting key 9ee47658c4be236ed239b758de2b031e
I20260619 08:55:16.515206 5560 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb848918df54ffeb1bffb4bb986c3f5" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38637 } }
I20260619 08:55:16.515314 5560 leader_election.cc:302] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beb848918df54ffeb1bffb4bb986c3f5; no voters:
I20260619 08:55:16.515585 5560 leader_election.cc:288] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:16.515627 5564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:16.515789 5564 raft_consensus.cc:686] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [term 1 LEADER]: Becoming Leader. State: Replica: beb848918df54ffeb1bffb4bb986c3f5, State: Running, Role: LEADER
I20260619 08:55:16.515897 5560 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:16.515928 5564 consensus_queue.cc:237] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb848918df54ffeb1bffb4bb986c3f5" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38637 } }
I20260619 08:55:16.516281 5564 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader beb848918df54ffeb1bffb4bb986c3f5. Latest consensus state: current_term: 1 leader_uuid: "beb848918df54ffeb1bffb4bb986c3f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb848918df54ffeb1bffb4bb986c3f5" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38637 } } }
I20260619 08:55:16.516355 5564 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:16.516449 5566 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beb848918df54ffeb1bffb4bb986c3f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb848918df54ffeb1bffb4bb986c3f5" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 38637 } } }
I20260619 08:55:16.516896 5566 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:16.516917 5574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:16.517519 5574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:16.517639 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:16.515454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46409 (local address 127.15.208.62:38637)
0619 08:55:16.515811 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:16.515820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:16.515850 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:55:16.515964 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:16.515972 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:16.516003 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:16.516196 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:16.517063 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:16.517067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:16.517067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:16.517073 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:16.517116 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:16.517131 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:16.517203 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:16.517378 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 08:55:16.517422 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":119,"threads_started":1}
I20260619 08:55:16.518714 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:16.519073 5574 catalog_manager.cc:1347] Generated new cluster ID: 1ca82026719a47bdb10ff1376e53b0e3
I20260619 08:55:16.519121 5574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:16.521410 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:16.520272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40657 (local address 127.15.208.62:38637)
0619 08:55:16.520305 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:16.520306 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:16.520417 (+ 111us) server_negotiation.cc:407] Connection header received
0619 08:55:16.520502 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:16.520503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:16.520508 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:16.520534 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:16.521203 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:16.521204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:16.521205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:16.521209 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:16.521222 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:16.521226 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:16.521252 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:16.521338 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 08:55:16.521351 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 08:55:16.540081 5574 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:16.540292 5574 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:16.548205 5574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beb848918df54ffeb1bffb4bb986c3f5: Generated new TSK 0
I20260619 08:55:16.548408 5574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:16.556136 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 5492
2026-06-19T08:55:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:16 dist-test-slave-p0h9 krb5kdc[5592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:16 dist-test-slave-p0h9 krb5kdc[5592](info): set up 2 sockets
Jun 19 08:55:16 dist-test-slave-p0h9 krb5kdc[5592](info): commencing operation
krb5kdc: starting...
W20260619 08:55:17.589447 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:17 dist-test-slave-p0h9 krb5kdc[5592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:55:17.708803 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:40869
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:17.789603 5605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:17.789921 5605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:17.789965 5605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:17.791213 5605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:17.791292 5605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:17.791317 5605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:17.791340 5605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:17.791369 5605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:17.792919 5605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:40869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:40869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:17.793220 5605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:17.793454 5605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:17.798317 5611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:17.798337 5613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:17.798317 5610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:17.798600 5605 server_base.cc:1034] running on GCE node
I20260619 08:55:17.798991 5605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:17.799342 5605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:17.800488 5605 hybrid_clock.cc:648] HybridClock initialized: now 1781859317800468 us; error 41 us; skew 500 ppm
Jun 19 08:55:17 dist-test-slave-p0h9 krb5kdc[5592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859317, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:17.803103 5605 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:55:17.803562 5605 webserver.cc:460] Webserver started at http://127.15.208.62:44265/ using document root <none> and password file <none>
I20260619 08:55:17.803754 5605 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:17.803794 5605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:17.803884 5605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:17.805022 5605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "bbceeeeb187d4607849ffdf5556eb972"
format_stamp: "Formatted at 2026-06-19 08:55:17 on dist-test-slave-p0h9"
server_key: "00af209741f11abad83419636b598735"
server_key_iv: "a55718db45e742f911baa3ab98f07a8c"
server_key_version: "encryptionkey@0"
I20260619 08:55:17.805346 5605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "bbceeeeb187d4607849ffdf5556eb972"
format_stamp: "Formatted at 2026-06-19 08:55:17 on dist-test-slave-p0h9"
server_key: "00af209741f11abad83419636b598735"
server_key_iv: "a55718db45e742f911baa3ab98f07a8c"
server_key_version: "encryptionkey@0"
I20260619 08:55:17.806622 5605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:55:17.807380 5620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:17.807598 5605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:55:17.807706 5605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "bbceeeeb187d4607849ffdf5556eb972"
format_stamp: "Formatted at 2026-06-19 08:55:17 on dist-test-slave-p0h9"
server_key: "00af209741f11abad83419636b598735"
server_key_iv: "a55718db45e742f911baa3ab98f07a8c"
server_key_version: "encryptionkey@0"
I20260619 08:55:17.807785 5605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:17.831565 5605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:17.834332 5605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:17.834474 5605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:17.839329 5605 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:40869
I20260619 08:55:17.839370 5673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:40869 every 8 connection(s)
I20260619 08:55:17.839882 5605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:17.840817 5674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:17.843094 5674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972: Bootstrap starting.
I20260619 08:55:17.843719 5674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:17.843961 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 5605
I20260619 08:55:17.844080 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:17.844102 5674 log.cc:824] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:17.844327 16192 external_mini_cluster.cc:1428] Setting key 2a850abd6bdb3090f21e33494173ad1f
I20260619 08:55:17.845019 5674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972: No bootstrap required, opened a new log
I20260619 08:55:17.847651 5674 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceeeeb187d4607849ffdf5556eb972" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 40869 } }
I20260619 08:55:17.847836 5674 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:17.847909 5674 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbceeeeb187d4607849ffdf5556eb972, State: Initialized, Role: FOLLOWER
I20260619 08:55:17.848043 5674 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceeeeb187d4607849ffdf5556eb972" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 40869 } }
I20260619 08:55:17.848117 5674 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:17.848145 5674 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:17.848232 5674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:17.848964 5674 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceeeeb187d4607849ffdf5556eb972" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 40869 } }
I20260619 08:55:17.849087 5674 leader_election.cc:302] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbceeeeb187d4607849ffdf5556eb972; no voters:
I20260619 08:55:17.849287 5674 leader_election.cc:288] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:17.849334 5679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:17.849476 5679 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [term 1 LEADER]: Becoming Leader. State: Replica: bbceeeeb187d4607849ffdf5556eb972, State: Running, Role: LEADER
I20260619 08:55:17.849583 5674 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:17.849566 5679 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceeeeb187d4607849ffdf5556eb972" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 40869 } }
I20260619 08:55:17.849912 5679 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbceeeeb187d4607849ffdf5556eb972. Latest consensus state: current_term: 1 leader_uuid: "bbceeeeb187d4607849ffdf5556eb972" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceeeeb187d4607849ffdf5556eb972" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 40869 } } }
I20260619 08:55:17.850006 5679 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:17.850226 5685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:17.850747 5685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:55:17 dist-test-slave-p0h9 krb5kdc[5592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:55:17.851506 5680 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbceeeeb187d4607849ffdf5556eb972" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceeeeb187d4607849ffdf5556eb972" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 40869 } } }
I20260619 08:55:17.852048 5680 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:17.853103 5685 catalog_manager.cc:1347] Generated new cluster ID: 461983e304ca4950909b8947d7fbe96b
I20260619 08:55:17.853183 5685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:17.860004 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:17.844931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47301 (local address 127.15.208.62:40869)
0619 08:55:17.845112 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:17.845122 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:17.845156 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:55:17.845572 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:17.845579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:17.845626 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:17.845767 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:17.846573 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:17.847471 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:17.848532 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:17.849090 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:17.849214 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:17.852476 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:17.852480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:17.852480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:17.852493 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:17.858309 (+ 5816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:17.858768 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:17.858771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:17.858771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:17.858874 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:17.859093 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:17.859094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:17.859094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:17.859378 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:17.859517 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:17.859661 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 08:55:17.859791 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260619 08:55:17.861084 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:17.869477 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:17.862515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42493 (local address 127.15.208.62:40869)
0619 08:55:17.862558 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:17.862560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:17.862622 (+ 62us) server_negotiation.cc:407] Connection header received
0619 08:55:17.862714 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:17.862715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:17.862724 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:17.862749 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:17.863382 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:17.864142 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:17.865149 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:17.865443 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:17.865480 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:17.866835 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:17.866836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:17.866837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:17.866843 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:17.868284 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:17.868631 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:17.868632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:17.868632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:17.868689 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:17.868928 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:17.868929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:17.868929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:17.869039 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:17.869103 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:17.869307 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 08:55:17.869384 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:55:17.870783 5685 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:17.871488 5685 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:17.873905 5685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbceeeeb187d4607849ffdf5556eb972: Generated new TSK 0
I20260619 08:55:17.874078 5685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:17.880128 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 5605
2026-06-19T08:55:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:17 dist-test-slave-p0h9 krb5kdc[5705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:17 dist-test-slave-p0h9 krb5kdc[5705](info): set up 2 sockets
Jun 19 08:55:17 dist-test-slave-p0h9 krb5kdc[5705](info): commencing operation
krb5kdc: starting...
W20260619 08:55:18.921375 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:19 dist-test-slave-p0h9 krb5kdc[5705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:55:19.037534 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:35133
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:35133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:19.118969 5719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:19.119184 5719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:19.119211 5719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:19.120352 5719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:19.120398 5719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:19.120412 5719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:19.120425 5719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:19.120478 5719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:19.122069 5719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:35133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:35133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:19.122344 5719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:19.122575 5719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:19.127609 5727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:19.127590 5724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:19.127615 5725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:19.127799 5719 server_base.cc:1034] running on GCE node
I20260619 08:55:19.128094 5719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:19.128391 5719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:19.129657 5719 hybrid_clock.cc:648] HybridClock initialized: now 1781859319129600 us; error 60 us; skew 500 ppm
Jun 19 08:55:19 dist-test-slave-p0h9 krb5kdc[5705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859319, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:19.132910 5719 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:55:19.133376 5719 webserver.cc:460] Webserver started at http://127.15.208.62:43439/ using document root <none> and password file <none>
I20260619 08:55:19.133646 5719 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:19.133718 5719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:19.133873 5719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:19.134977 5719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "d245b36a630949e2b06cc405110cc959"
format_stamp: "Formatted at 2026-06-19 08:55:19 on dist-test-slave-p0h9"
server_key: "e3d4453d63d81359e4b8395daf92a77f"
server_key_iv: "183a9fba85a45e0ac7d0e8f19b953b23"
server_key_version: "encryptionkey@0"
I20260619 08:55:19.135355 5719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "d245b36a630949e2b06cc405110cc959"
format_stamp: "Formatted at 2026-06-19 08:55:19 on dist-test-slave-p0h9"
server_key: "e3d4453d63d81359e4b8395daf92a77f"
server_key_iv: "183a9fba85a45e0ac7d0e8f19b953b23"
server_key_version: "encryptionkey@0"
I20260619 08:55:19.136792 5719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:55:19.137593 5734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:19.137769 5719 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:55:19.137893 5719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "d245b36a630949e2b06cc405110cc959"
format_stamp: "Formatted at 2026-06-19 08:55:19 on dist-test-slave-p0h9"
server_key: "e3d4453d63d81359e4b8395daf92a77f"
server_key_iv: "183a9fba85a45e0ac7d0e8f19b953b23"
server_key_version: "encryptionkey@0"
I20260619 08:55:19.137981 5719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:19.147455 5719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:19.149821 5719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:19.149986 5719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:19.153924 5719 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:35133
I20260619 08:55:19.153971 5786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:35133 every 8 connection(s)
I20260619 08:55:19.154326 5719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:19.155151 5787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:19.157441 5787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959: Bootstrap starting.
I20260619 08:55:19.157958 5787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:19.158197 5787 log.cc:824] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:19.158830 5787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959: No bootstrap required, opened a new log
I20260619 08:55:19.160370 5787 raft_consensus.cc:348] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d245b36a630949e2b06cc405110cc959" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35133 } }
I20260619 08:55:19.160504 5787 raft_consensus.cc:374] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:19.160527 5787 raft_consensus.cc:729] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d245b36a630949e2b06cc405110cc959, State: Initialized, Role: FOLLOWER
I20260619 08:55:19.160663 5787 consensus_queue.cc:260] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d245b36a630949e2b06cc405110cc959" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35133 } }
I20260619 08:55:19.160804 5787 raft_consensus.cc:388] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:19.160832 5787 raft_consensus.cc:482] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:19.160866 5787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:19.161675 5787 raft_consensus.cc:504] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d245b36a630949e2b06cc405110cc959" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35133 } }
I20260619 08:55:19.161769 5787 leader_election.cc:302] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d245b36a630949e2b06cc405110cc959; no voters:
I20260619 08:55:19.161895 5787 leader_election.cc:288] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:19.161964 5790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:19.162130 5790 raft_consensus.cc:686] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [term 1 LEADER]: Becoming Leader. State: Replica: d245b36a630949e2b06cc405110cc959, State: Running, Role: LEADER
I20260619 08:55:19.162173 5787 sys_catalog.cc:564] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:19.162209 5790 consensus_queue.cc:237] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d245b36a630949e2b06cc405110cc959" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35133 } }
I20260619 08:55:19.162211 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 5719
I20260619 08:55:19.162328 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:19.162509 16192 external_mini_cluster.cc:1428] Setting key c9fe6f1749f23973ce92137785b88d55
I20260619 08:55:19.162559 5790 sys_catalog.cc:455] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d245b36a630949e2b06cc405110cc959. Latest consensus state: current_term: 1 leader_uuid: "d245b36a630949e2b06cc405110cc959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d245b36a630949e2b06cc405110cc959" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35133 } } }
I20260619 08:55:19.162616 5790 sys_catalog.cc:458] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:19.162660 5791 sys_catalog.cc:455] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d245b36a630949e2b06cc405110cc959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d245b36a630949e2b06cc405110cc959" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35133 } } }
I20260619 08:55:19.162719 5791 sys_catalog.cc:458] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:19.162824 5796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:19.163375 5796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:19.165465 5796 catalog_manager.cc:1347] Generated new cluster ID: e946d4b8089b4203bfaca57de6169b70
I20260619 08:55:19.165520 5796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:19.168560 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:19.163365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49531 (local address 127.15.208.62:35133)
0619 08:55:19.163983 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:19.163992 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:19.164022 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:55:19.164136 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:19.164144 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:19.164197 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:19.164332 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:19.164598 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:19.165806 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:19.166606 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:19.167041 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:19.167169 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:19.167878 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:19.167882 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:19.167882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:19.168082 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:19.168111 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:19.168123 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:19.168198 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:19.168275 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 08:55:19.168338 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":70,"threads_started":1}
I20260619 08:55:19.169552 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:19.179921 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:19.170910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59237 (local address 127.15.208.62:35133)
0619 08:55:19.170949 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:19.170951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:19.176959 (+ 6008us) server_negotiation.cc:407] Connection header received
0619 08:55:19.177048 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:19.177049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:19.177073 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:19.177148 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:19.177299 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:19.177813 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:19.178545 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:19.178802 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:19.178884 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:19.179610 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:19.179611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:19.179612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:19.179616 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:19.179631 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:19.179635 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:19.179683 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:19.179737 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 08:55:19.179762 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:55:19.199239 5796 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:19.199800 5796 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:19.203814 5796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d245b36a630949e2b06cc405110cc959: Generated new TSK 0
I20260619 08:55:19.204005 5796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:19.207798 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 5719
2026-06-19T08:55:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:19 dist-test-slave-p0h9 krb5kdc[5818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:19 dist-test-slave-p0h9 krb5kdc[5818](info): set up 2 sockets
Jun 19 08:55:19 dist-test-slave-p0h9 krb5kdc[5818](info): commencing operation
krb5kdc: starting...
W20260619 08:55:20.297465 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:20 dist-test-slave-p0h9 krb5kdc[5818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:55:20.417888 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:35367
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:43815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:35367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:20.507086 5831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:20.507295 5831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:20.507318 5831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:20.508381 5831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:20.508415 5831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:20.508430 5831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:20.508442 5831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:20.508455 5831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:20.509894 5831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:43815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:35367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:35367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:20.510082 5831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:20.510248 5831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:20.514959 5837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:20.514973 5836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:20.514978 5839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:20.515267 5831 server_base.cc:1034] running on GCE node
I20260619 08:55:20.515456 5831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:20.515691 5831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:20.516804 5831 hybrid_clock.cc:648] HybridClock initialized: now 1781859320516790 us; error 30 us; skew 500 ppm
Jun 19 08:55:20 dist-test-slave-p0h9 krb5kdc[5818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859320, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:20.519311 5831 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:55:20.519773 5831 webserver.cc:460] Webserver started at http://127.15.208.62:42395/ using document root <none> and password file <none>
I20260619 08:55:20.519999 5831 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:20.520063 5831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:20.520233 5831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:20.521345 5831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "d89a59f0f2c84535b3ac6308590aaeed"
format_stamp: "Formatted at 2026-06-19 08:55:20 on dist-test-slave-p0h9"
server_key: "f9e5c962863641269940828ae8f80d35"
server_key_iv: "d1354851ac49da56c1cd737b2173721c"
server_key_version: "encryptionkey@0"
I20260619 08:55:20.521739 5831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "d89a59f0f2c84535b3ac6308590aaeed"
format_stamp: "Formatted at 2026-06-19 08:55:20 on dist-test-slave-p0h9"
server_key: "f9e5c962863641269940828ae8f80d35"
server_key_iv: "d1354851ac49da56c1cd737b2173721c"
server_key_version: "encryptionkey@0"
I20260619 08:55:20.523078 5831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:55:20.523808 5846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:20.523970 5831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:55:20.524073 5831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "d89a59f0f2c84535b3ac6308590aaeed"
format_stamp: "Formatted at 2026-06-19 08:55:20 on dist-test-slave-p0h9"
server_key: "f9e5c962863641269940828ae8f80d35"
server_key_iv: "d1354851ac49da56c1cd737b2173721c"
server_key_version: "encryptionkey@0"
I20260619 08:55:20.524158 5831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:20.527509 5831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:20.529515 5831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:20.529671 5831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:20.533566 5831 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:35367
I20260619 08:55:20.533648 5899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:35367 every 8 connection(s)
I20260619 08:55:20.533990 5831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:20.534948 5900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:20.537313 5900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed: Bootstrap starting.
I20260619 08:55:20.537892 5900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:20.538184 5900 log.cc:824] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:20.538851 5900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed: No bootstrap required, opened a new log
I20260619 08:55:20.540428 5900 raft_consensus.cc:348] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a59f0f2c84535b3ac6308590aaeed" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35367 } }
I20260619 08:55:20.540560 5900 raft_consensus.cc:374] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:55:20.540584 5900 raft_consensus.cc:729] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d89a59f0f2c84535b3ac6308590aaeed, State: Initialized, Role: FOLLOWER
I20260619 08:55:20.540814 5900 consensus_queue.cc:260] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a59f0f2c84535b3ac6308590aaeed" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35367 } }
I20260619 08:55:20.540884 5900 raft_consensus.cc:388] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:55:20.540906 5900 raft_consensus.cc:482] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:55:20.540973 5900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:20.541811 5900 raft_consensus.cc:504] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a59f0f2c84535b3ac6308590aaeed" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35367 } }
I20260619 08:55:20.541922 5900 leader_election.cc:302] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d89a59f0f2c84535b3ac6308590aaeed; no voters:
I20260619 08:55:20.542110 5900 leader_election.cc:288] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:55:20.542173 5903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:20.542354 5903 raft_consensus.cc:686] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [term 1 LEADER]: Becoming Leader. State: Replica: d89a59f0f2c84535b3ac6308590aaeed, State: Running, Role: LEADER
I20260619 08:55:20.542378 5900 sys_catalog.cc:564] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:20.542456 5903 consensus_queue.cc:237] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a59f0f2c84535b3ac6308590aaeed" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35367 } }
I20260619 08:55:20.542826 5904 sys_catalog.cc:455] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d89a59f0f2c84535b3ac6308590aaeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a59f0f2c84535b3ac6308590aaeed" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35367 } } }
I20260619 08:55:20.542972 5904 sys_catalog.cc:458] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:20.543089 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 5831
I20260619 08:55:20.543208 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:20.543227 5912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:20.543434 16192 external_mini_cluster.cc:1428] Setting key d3cfe348ac1c6b0cb36aa8a0c2d2271f
I20260619 08:55:20.543759 5912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:20.542829 5905 sys_catalog.cc:455] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [sys.catalog]: SysCatalogTable state changed. Reason: New leader d89a59f0f2c84535b3ac6308590aaeed. Latest consensus state: current_term: 1 leader_uuid: "d89a59f0f2c84535b3ac6308590aaeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a59f0f2c84535b3ac6308590aaeed" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 35367 } } }
I20260619 08:55:20.544202 5905 sys_catalog.cc:458] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:20.545784 5912 catalog_manager.cc:1347] Generated new cluster ID: 2f33b9becbfa4973aa895dc17101a88f
I20260619 08:55:20.545864 5912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:20.546864 5922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:20.543932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40811 (local address 127.15.208.62:35367)
0619 08:55:20.545456 (+ 1524us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:20.545465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:20.545499 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:55:20.545622 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:20.545629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:20.545661 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:20.545840 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:20.546507 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:20.546510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:20.546511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:20.546514 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:20.546547 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:20.546558 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:20.546638 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:20.546708 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 08:55:20.546741 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1456,"thread_start_us":86,"threads_started":1}
I20260619 08:55:20.547725 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:20.550031 5922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:20.549195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60779 (local address 127.15.208.62:35367)
0619 08:55:20.549230 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:20.549231 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:20.549242 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:55:20.549300 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:20.549301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:20.549304 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:20.549348 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:20.549859 (+ 511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:20.549860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:20.549860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:55:20.549863 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:20.549876 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:55:20.549880 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:20.549925 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:20.549980 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 08:55:20.549990 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:55:20.557515 5912 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:20.557708 5912 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:20.565742 5912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d89a59f0f2c84535b3ac6308590aaeed: Generated new TSK 0
I20260619 08:55:20.565928 5912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:55:20.574523 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 5831
2026-06-19T08:55:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1367 ms)
[----------] 18 tests from AuthTokenIssuingTest (23812 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:20 dist-test-slave-p0h9 krb5kdc[5931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:20 dist-test-slave-p0h9 krb5kdc[5931](info): set up 2 sockets
Jun 19 08:55:20 dist-test-slave-p0h9 krb5kdc[5931](info): commencing operation
krb5kdc: starting...
W20260619 08:55:21.601462 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:21 dist-test-slave-p0h9 krb5kdc[5931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:55:21.717810 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:34891
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:34891,127.15.208.61:45951,127.15.208.60:41567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:21.797282 5944 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:55:21.797523 5944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:21.797542 5944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:21.797562 5944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:21.798625 5944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:21.798660 5944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:21.798674 5944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:21.798686 5944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:21.798698 5944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:21.800076 5944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:34891,127.15.208.61:45951,127.15.208.60:41567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:21.800267 5944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:21.800434 5944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:21.805337 5949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:21.805343 5950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:21.805425 5952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:21.805748 5944 server_base.cc:1034] running on GCE node
I20260619 08:55:21.806026 5944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:21.806351 5944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:21.807507 5944 hybrid_clock.cc:648] HybridClock initialized: now 1781859321807483 us; error 39 us; skew 500 ppm
Jun 19 08:55:21 dist-test-slave-p0h9 krb5kdc[5931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:21.809958 5944 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:55:21.810454 5944 webserver.cc:460] Webserver started at http://127.15.208.62:34167/ using document root <none> and password file <none>
I20260619 08:55:21.810684 5944 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:21.810752 5944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:21.810911 5944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:21.811983 5944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a"
format_stamp: "Formatted at 2026-06-19 08:55:21 on dist-test-slave-p0h9"
server_key: "624e7c7967d6deac0358debc33abeff9"
server_key_iv: "b287d433bb99b4780ad1ef46a704350a"
server_key_version: "encryptionkey@0"
I20260619 08:55:21.812361 5944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a"
format_stamp: "Formatted at 2026-06-19 08:55:21 on dist-test-slave-p0h9"
server_key: "624e7c7967d6deac0358debc33abeff9"
server_key_iv: "b287d433bb99b4780ad1ef46a704350a"
server_key_version: "encryptionkey@0"
I20260619 08:55:21.813794 5944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:55:21.814500 5959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:21.814724 5944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:21.814822 5944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a"
format_stamp: "Formatted at 2026-06-19 08:55:21 on dist-test-slave-p0h9"
server_key: "624e7c7967d6deac0358debc33abeff9"
server_key_iv: "b287d433bb99b4780ad1ef46a704350a"
server_key_version: "encryptionkey@0"
I20260619 08:55:21.814901 5944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:21.831166 5944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:21.833249 5944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:21.833442 5944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:21.838007 5944 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:34891
I20260619 08:55:21.838038 6011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:34891 every 8 connection(s)
I20260619 08:55:21.838392 5944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:21.839320 6012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:21.840866 6012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:21.842459 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 5944
I20260619 08:55:21.842574 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:21.842758 16192 external_mini_cluster.cc:1428] Setting key 486456534dfcf4862972f4961981c5d3
Jun 19 08:55:21 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.208.61@KRBTEST.COM; defaulting to no policy
I20260619 08:55:21.851821 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.841475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:34891 (local address 127.0.0.1:56530)
0619 08:55:21.841620 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:21.841652 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:21.842056 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:21.842350 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:21.842352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:21.842684 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:21.842882 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:21.842886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:21.844406 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:21.844407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:21.845522 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:21.845524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:21.846292 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:21.846293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:21.846346 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:21.846902 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:21.846910 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:21.848262 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:21.850362 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:21.850369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:21.850370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:21.850558 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:21.850828 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:21.850828 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:21.850829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:21.850909 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:21.851360 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:21.851365 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:21.851421 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 08:55:21.851497 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:55:21.851626 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260619 08:55:21.851824 6015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.841625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.15.208.62:34891)
0619 08:55:21.841753 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:21.841756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:21.841781 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:55:21.842211 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:21.842213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:21.842228 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:21.842341 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:21.842972 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:21.844282 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:21.845602 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:21.846186 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:21.846302 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:21.848348 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:21.848357 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:21.848358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:21.848366 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:21.850283 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:21.850645 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:21.850649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:21.850649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:21.850753 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:21.850978 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:21.850979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:21.850979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:21.851285 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:21.851376 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:21.851541 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 08:55:21.851683 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260619 08:55:21.852713 6012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:21.853153 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.853041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:45951 (local address 127.0.0.1:51884)
0619 08:55:21.853075 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:21.853126 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260619 08:55:21.853336 5960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.208.61:45951: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.61@KRBTEST.COM" created.
I20260619 08:55:21.853811 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.853736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:45951 (local address 127.0.0.1:51886)
0619 08:55:21.853766 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:21.853790 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260619 08:55:21.853915 6012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.61:45951: Network error: Client connection negotiation failed: client connection to 127.15.208.61:45951: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab.
Entry for principal kudu/127.15.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab.
I20260619 08:55:21.867163 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.61:45951
--webserver_interface=127.15.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab
--principal=kudu/127.15.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:34891,127.15.208.61:45951,127.15.208.60:41567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:55:21.903230 6012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } attempt: 1
I20260619 08:55:21.903752 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.903610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:45951 (local address 127.0.0.1:51896)
0619 08:55:21.903682 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:21.903723 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 08:55:21.904083 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.903989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:45951 (local address 127.0.0.1:51900)
0619 08:55:21.904033 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:21.904059 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 08:55:21.904174 6012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.61:45951: Network error: Client connection negotiation failed: client connection to 127.15.208.61:45951: connect: Connection refused (error 111)
W20260619 08:55:21.947434 6019 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:55:21.947676 6019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:21.947695 6019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:21.947716 6019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:21.948874 6019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:21.948912 6019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:21.948926 6019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:21.948938 6019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:21.948951 6019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:21.950433 6019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:34891,127.15.208.61:45951,127.15.208.60:41567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab
--principal=kudu/127.15.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.61:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:21.950675 6019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:21.950913 6019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:21.955606 6027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:21.955674 6025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:21.955785 6019 server_base.cc:1034] running on GCE node
W20260619 08:55:21.955583 6024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:21.956142 6019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:21.956430 6019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:21.957636 6019 hybrid_clock.cc:648] HybridClock initialized: now 1781859321957629 us; error 36 us; skew 500 ppm
Jun 19 08:55:21 dist-test-slave-p0h9 krb5kdc[5931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:21.960224 6019 init.cc:377] Logged in from keytab as kudu/127.15.208.61@KRBTEST.COM (short username kudu)
I20260619 08:55:21.960601 6019 webserver.cc:460] Webserver started at http://127.15.208.61:37071/ using document root <none> and password file <none>
I20260619 08:55:21.960851 6019 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:21.960899 6019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:21.960988 6019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:21.962044 6019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/data/instance:
uuid: "adbf4a7c389a4c6db33babd7445a96a5"
format_stamp: "Formatted at 2026-06-19 08:55:21 on dist-test-slave-p0h9"
server_key: "a386f027d938ab23ae6deb845d2d684a"
server_key_iv: "e8cc4ae5c09daa96524a3bdac4ab32f5"
server_key_version: "encryptionkey@0"
I20260619 08:55:21.962342 6019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/wal/instance:
uuid: "adbf4a7c389a4c6db33babd7445a96a5"
format_stamp: "Formatted at 2026-06-19 08:55:21 on dist-test-slave-p0h9"
server_key: "a386f027d938ab23ae6deb845d2d684a"
server_key_iv: "e8cc4ae5c09daa96524a3bdac4ab32f5"
server_key_version: "encryptionkey@0"
I20260619 08:55:21.963578 6019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:55:21.964229 6034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:21.964380 6019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:55:21.964464 6019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/wal
uuid: "adbf4a7c389a4c6db33babd7445a96a5"
format_stamp: "Formatted at 2026-06-19 08:55:21 on dist-test-slave-p0h9"
server_key: "a386f027d938ab23ae6deb845d2d684a"
server_key_iv: "e8cc4ae5c09daa96524a3bdac4ab32f5"
server_key_version: "encryptionkey@0"
I20260619 08:55:21.964519 6019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:21.972308 6012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } attempt: 2
I20260619 08:55:21.972761 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.972575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:45951 (local address 127.0.0.1:51914)
0619 08:55:21.972639 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:21.972677 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:55:21.973140 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.973053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:45951 (local address 127.0.0.1:51920)
0619 08:55:21.973098 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:21.973123 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:55:21.973245 6012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.61:45951: Network error: Client connection negotiation failed: client connection to 127.15.208.61:45951: connect: Connection refused (error 111)
I20260619 08:55:21.977990 6019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:21.980183 6019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:21.980352 6019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:21.984290 6019 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.61:45951
I20260619 08:55:21.984392 6086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.61:45951 every 8 connection(s)
I20260619 08:55:21.984704 6019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
I20260619 08:55:21.985730 6087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:21.987226 6087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:21.991729 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 6019
I20260619 08:55:21.991811 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-1/wal/instance
I20260619 08:55:21.991972 16192 external_mini_cluster.cc:1428] Setting key 89acda0df31281098447c1ae77074260
Jun 19 08:55:21 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:55:21.997081 6015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.987929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56542 (local address 127.15.208.62:34891)
0619 08:55:21.987982 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:21.987984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:21.988179 (+ 195us) server_negotiation.cc:407] Connection header received
0619 08:55:21.988604 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:21.988605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:21.988615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:21.988667 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:21.989264 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:21.989728 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:21.990656 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:21.990842 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:21.990892 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:21.993381 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:21.993383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:21.993384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:21.993391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:21.995063 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:21.995444 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:21.995445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:21.995446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:21.995519 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:21.996138 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:21.996139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:21.996139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:21.996329 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:21.996422 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:21.996870 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 08:55:21.996978 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:55:21.997203 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.987883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:34891 (local address 127.0.0.1:56542)
0619 08:55:21.988056 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:21.988096 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:21.988526 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:21.988765 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:21.988769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:21.989009 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:21.989192 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:21.989195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:21.989807 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:21.989807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:21.990580 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:21.990582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:21.990930 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:21.990931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:21.991021 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:21.991573 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:21.991581 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:21.993257 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:21.995148 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:21.995154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:21.995155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:21.995366 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:21.995590 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:21.995590 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:21.995590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:21.995705 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:21.996421 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:21.996429 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:21.996682 (+ 253us) client_negotiation.cc:769] Sending connection context
0619 08:55:21.996809 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 08:55:21.996977 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260619 08:55:21.997673 6087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.208.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.60@KRBTEST.COM" created.
Jun 19 08:55:21 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
I20260619 08:55:22.007334 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.997956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:45951 (local address 127.0.0.1:51922)
0619 08:55:21.998012 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:21.998023 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:21.998066 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:21.998356 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:21.998356 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:21.998463 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:21.998514 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:21.998515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:21.999840 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:21.999841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.000325 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.000326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.001018 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.001019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.001075 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.001444 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:22.001448 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:22.003957 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:22.006280 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.006281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.006281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.006446 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.006676 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.006677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.006677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.006727 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.007077 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:22.007078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:22.007114 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 08:55:22.007162 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:55:22.007256 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 08:55:22.007469 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:21.998034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.15.208.61:45951)
0619 08:55:21.998196 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:21.998201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:21.998227 (+ 26us) server_negotiation.cc:407] Connection header received
0619 08:55:21.998250 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:21.998252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:21.998266 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:21.998348 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:21.998589 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:21.999743 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.000408 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.000940 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.001010 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.004063 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.004071 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.004071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.004077 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.006140 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.006533 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.006536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.006536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.006604 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.006793 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.006794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.006794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.007000 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.007089 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.007202 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.007341 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260619 08:55:22.008312 6087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:22.008698 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.008548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42348)
0619 08:55:22.008602 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.008665 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:55:22.008882 6036 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
I20260619 08:55:22.009397 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.009279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42352)
0619 08:55:22.009338 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.009370 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 08:55:22.009564 6087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:41567: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab.
Entry for principal kudu/127.15.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab.
I20260619 08:55:22.017774 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.60:41567
--webserver_interface=127.15.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab
--principal=kudu/127.15.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:34891,127.15.208.61:45951,127.15.208.60:41567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:55:22.058821 6087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } attempt: 1
I20260619 08:55:22.059399 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.059240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42356)
0619 08:55:22.059315 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.059363 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 08:55:22.059793 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.059701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42368)
0619 08:55:22.059743 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.059771 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 08:55:22.059911 6087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:41567: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
I20260619 08:55:22.064391 6012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } attempt: 3
Jun 19 08:55:22 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
I20260619 08:55:22.072508 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.064780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.15.208.61:45951)
0619 08:55:22.064842 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.064844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.064856 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:55:22.064947 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.064948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.064960 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.065035 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.065359 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.065850 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.066582 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.066868 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.066968 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.069536 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.069537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.069538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.069546 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.071273 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.071690 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.071692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.071692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.071753 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.071969 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.071970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.071971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.072103 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.072166 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.072333 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.072422 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:55:22.072546 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.064711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:45951 (local address 127.0.0.1:51934)
0619 08:55:22.064778 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.064812 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:22.064866 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:22.065040 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:22.065041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:22.065208 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:22.065274 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.065276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.065943 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.065944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.066495 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.066497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.066993 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.066994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.067092 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.067619 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:22.067625 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:22.069419 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:22.071364 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.071365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.071366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.071605 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.071832 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.071833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.071833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.071903 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.072173 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:22.072174 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:22.072225 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 08:55:22.072298 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:55:22.072415 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 08:55:22.073271 6012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:22.073559 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.073482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42384)
0619 08:55:22.073518 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.073539 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 08:55:22.073863 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.073721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42394)
0619 08:55:22.073826 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.073843 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":80,"threads_started":1}
W20260619 08:55:22.073922 6012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:41567: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
W20260619 08:55:22.100443 6094 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:55:22.100668 6094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:22.100700 6094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:22.100786 6094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:22.101886 6094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:22.101958 6094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:22.101984 6094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:22.102012 6094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:22.102033 6094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:22.103440 6094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:34891,127.15.208.61:45951,127.15.208.60:41567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab
--principal=kudu/127.15.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.60:41567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:22.103704 6094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:22.103915 6094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:55:22.105172 6012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } attempt: 1
I20260619 08:55:22.105703 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.105496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42398)
0619 08:55:22.105590 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.105645 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:55:22.106050 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.105954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42414)
0619 08:55:22.106003 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.106030 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 08:55:22.106395 6012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:41567: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
W20260619 08:55:22.109027 6101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:22.109062 6100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:22.109022 6103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:22.109318 6094 server_base.cc:1034] running on GCE node
I20260619 08:55:22.109560 6094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:22.109859 6094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:22.111023 6094 hybrid_clock.cc:648] HybridClock initialized: now 1781859322111014 us; error 39 us; skew 500 ppm
Jun 19 08:55:22 dist-test-slave-p0h9 krb5kdc[5931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:22.113446 6094 init.cc:377] Logged in from keytab as kudu/127.15.208.60@KRBTEST.COM (short username kudu)
I20260619 08:55:22.113879 6094 webserver.cc:460] Webserver started at http://127.15.208.60:36581/ using document root <none> and password file <none>
I20260619 08:55:22.114099 6094 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:22.114171 6094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:22.114323 6094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:22.115367 6094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/data/instance:
uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3"
format_stamp: "Formatted at 2026-06-19 08:55:22 on dist-test-slave-p0h9"
server_key: "d57df7716b065e1f0d1236a19af3789f"
server_key_iv: "912a8be5bcac3a2b20b1045be05dd011"
server_key_version: "encryptionkey@0"
I20260619 08:55:22.115731 6094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/wal/instance:
uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3"
format_stamp: "Formatted at 2026-06-19 08:55:22 on dist-test-slave-p0h9"
server_key: "d57df7716b065e1f0d1236a19af3789f"
server_key_iv: "912a8be5bcac3a2b20b1045be05dd011"
server_key_version: "encryptionkey@0"
I20260619 08:55:22.117089 6094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:55:22.117913 6110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:22.118081 6094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:55:22.118177 6094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/wal
uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3"
format_stamp: "Formatted at 2026-06-19 08:55:22 on dist-test-slave-p0h9"
server_key: "d57df7716b065e1f0d1236a19af3789f"
server_key_iv: "912a8be5bcac3a2b20b1045be05dd011"
server_key_version: "encryptionkey@0"
I20260619 08:55:22.118260 6094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:22.128172 6087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } attempt: 2
I20260619 08:55:22.128646 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.128488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42420)
0619 08:55:22.128554 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.128618 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 08:55:22.129019 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.128922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42434)
0619 08:55:22.128969 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.128994 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:55:22.129108 6087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:41567: Network error: Client connection negotiation failed: client connection to 127.15.208.60:41567: connect: Connection refused (error 111)
I20260619 08:55:22.142894 6094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:22.144898 6094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:22.145063 6094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:22.149089 6094 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.60:41567
I20260619 08:55:22.149127 6162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.60:41567 every 8 connection(s)
I20260619 08:55:22.149488 6094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
I20260619 08:55:22.150548 6163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:22.152069 6163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:22.152347 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 6094
I20260619 08:55:22.152447 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781859057751076-16192-0/minicluster-data/master-2/wal/instance
I20260619 08:55:22.152635 16192 external_mini_cluster.cc:1428] Setting key ff57dd5b412c743527381c8bb0d952b5
Jun 19 08:55:22 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
Jun 19 08:55:22 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
I20260619 08:55:22.162220 6015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.152880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56558 (local address 127.15.208.62:34891)
0619 08:55:22.152940 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.152943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.153359 (+ 416us) server_negotiation.cc:407] Connection header received
0619 08:55:22.154081 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.154082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.154099 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.154180 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.154891 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.155378 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.156265 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.156472 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.156550 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.159033 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.159035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.159035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.159044 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.160732 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.161152 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.161154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.161155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.161217 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.161495 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.161496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.161496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.161609 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.161701 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.162011 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.162106 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:55:22.162345 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.152873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:34891 (local address 127.0.0.1:56558)
0619 08:55:22.153208 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.153247 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:22.153890 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:22.154268 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:22.154272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:22.154623 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:22.154791 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.154796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.155465 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.155466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.156162 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.156164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.156576 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.156577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.156700 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.157294 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:22.157304 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:22.158943 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:22.160824 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.160831 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.160832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.161078 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.161288 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.161289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.161289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.161424 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.161703 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:22.161709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:22.161865 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 08:55:22.161982 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 08:55:22.162105 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":91,"threads_started":1}
I20260619 08:55:22.162796 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.153044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42446 (local address 127.15.208.60:41567)
0619 08:55:22.153204 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.153213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.153248 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:55:22.153887 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.153897 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.153922 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.154024 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.154894 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.155691 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.156444 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.156849 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.156935 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.159535 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.159542 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.159542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.159549 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.161310 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.161679 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.161682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.161682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.161740 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.161941 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.161942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.161942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.162124 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.162264 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.162604 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.162692 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":102,"threads_started":1}
I20260619 08:55:22.163190 6163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:55:22 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
Jun 19 08:55:22 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:55:22.170586 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.163556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:45951 (local address 127.0.0.1:51948)
0619 08:55:22.163606 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.163615 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:22.163650 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:22.163789 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:22.163790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:22.163897 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:22.163934 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.163935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.164750 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.164751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.165416 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.165417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.165890 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.165891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.165963 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.166346 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:22.166350 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:22.167834 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:22.169567 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.169568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.169568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.169760 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.170001 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.170002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.170002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.170053 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.170368 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:22.170369 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:22.170416 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 08:55:22.170464 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:55:22.170528 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 08:55:22.170737 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.163618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51948 (local address 127.15.208.61:45951)
0619 08:55:22.163681 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.163683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.163692 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:55:22.163718 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.163720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.163731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.163793 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.163984 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.164655 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.165490 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.165815 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.165888 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.167911 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.167912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.167912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.167919 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.169491 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.169847 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.169848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.169849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.169928 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.170117 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.170118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.170119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.170283 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.170372 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.170512 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.170582 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:55:22.171116 6163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:22.172109 6015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.163622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56564 (local address 127.15.208.62:34891)
0619 08:55:22.163833 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.163834 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.163839 (+ 5us) server_negotiation.cc:407] Connection header received
0619 08:55:22.163854 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.163854 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.163860 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.163916 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.164482 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.165144 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.165985 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.166242 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.166308 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.169058 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.169059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.169060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.169066 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.170734 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.171135 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.171136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.171136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.171196 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.171425 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.171426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.171427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.171571 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.171682 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.171833 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.171926 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
Jun 19 08:55:22 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
Jun 19 08:55:22 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
I20260619 08:55:22.178174 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.171370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42448)
0619 08:55:22.171706 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.171715 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:22.171775 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:22.171962 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:22.171963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:22.172095 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:22.172126 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.172127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.173004 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.173004 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.173629 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.173631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.174062 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.174063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.174111 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.174429 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:22.174433 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:22.175576 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:22.177166 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.177166 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.177167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.177371 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.177555 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.177556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.177557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.177622 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.177902 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:22.177902 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:22.177945 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 08:55:22.178016 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:55:22.178114 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324}
I20260619 08:55:22.178287 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.171436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.15.208.60:41567)
0619 08:55:22.171472 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.171474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.171762 (+ 288us) server_negotiation.cc:407] Connection header received
0619 08:55:22.171878 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.171879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.171889 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.171943 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.172302 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.172928 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.173706 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.173997 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.174054 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.175658 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.175659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.175660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.175666 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.177058 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.177447 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.177448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.177448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.177492 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.177699 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.177700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.177700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.177836 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.177885 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.178068 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.178172 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":18}
I20260619 08:55:22.180019 6163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3: Bootstrap starting.
I20260619 08:55:22.180414 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.172675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51958 (local address 127.15.208.61:45951)
0619 08:55:22.172707 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.172709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.172717 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:55:22.172800 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.172801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.172809 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.172854 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.173392 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.173902 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.174776 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.174968 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.175047 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.177565 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.177566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.177567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.177573 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.179300 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.179639 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.179640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.179640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.179698 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.179877 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.179878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.179878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.179998 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.180170 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.180245 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.180314 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:55:22.181183 6163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:22.181473 6163 log.cc:824] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:22.181559 6012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } attempt: 2
I20260619 08:55:22.181910 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:22.182379 6163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3: No bootstrap required, opened a new log
I20260619 08:55:22.182895 5986 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56564
I20260619 08:55:22.184571 6163 raft_consensus.cc:348] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } }
I20260619 08:55:22.184777 6163 raft_consensus.cc:729] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d20a65bcd62d4aa7a0f120d0021ce8e3, State: Initialized, Role: FOLLOWER
I20260619 08:55:22.184860 6163 consensus_queue.cc:260] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } }
I20260619 08:55:22.185245 6163 sys_catalog.cc:564] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:22.185204 6169 sys_catalog.cc:455] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } } }
I20260619 08:55:22.185308 6169 sys_catalog.cc:458] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [sys.catalog]: This master's current role is: FOLLOWER
Jun 19 08:55:22 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
W20260619 08:55:22.187045 6180 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:55:22.187098 6180 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:55:22.187474 6094 master_runner.cc:186] Error getting master registration for 127.15.208.62:34891: OK, instance_id { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" instance_seqno: 1781859321836875 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:55:22.189625 6094 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:55:22.189934 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.181870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42454 (local address 127.15.208.60:41567)
0619 08:55:22.182697 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.182700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.182708 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:55:22.182731 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.182732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.182741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.182795 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.183115 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.183568 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.184363 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.184562 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.184640 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.186728 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.186730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.186730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.186736 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.188545 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.189017 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.189018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.189018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.189079 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.189333 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.189334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.189334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.189497 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.189567 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.189773 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.189851 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806}
I20260619 08:55:22.189965 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.181824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42454)
0619 08:55:22.181869 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.181878 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:22.181926 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:22.182805 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:22.182806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:22.182964 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:22.183032 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.183033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.183660 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.183661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.184266 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.184268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.184638 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.184639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.184710 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.185212 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:22.185217 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:22.186646 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:22.188675 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.188676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.188677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.188936 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.189180 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.189181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.189181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.189244 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.189593 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:22.189594 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:22.189670 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 08:55:22.189746 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:55:22.189850 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 08:55:22.191438 6012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a: Bootstrap starting.
I20260619 08:55:22.192224 6012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:22.192520 6012 log.cc:824] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:22.193388 6012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a: No bootstrap required, opened a new log
I20260619 08:55:22.195282 6012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } }
I20260619 08:55:22.195411 6012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee6c5b9656e49dc8e0b435b0c8c561a, State: Initialized, Role: FOLLOWER
I20260619 08:55:22.195513 6012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } }
I20260619 08:55:22.195818 6012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:22.195842 5986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 08:55:22.195889 5986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:22.195776 6183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } } }
I20260619 08:55:22.196043 6183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:22.196785 5986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } }
I20260619 08:55:22.197070 5986 leader_election.cc:288] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [CANDIDATE]: Term 1 election: Requested vote from peers adbf4a7c389a4c6db33babd7445a96a5 (127.15.208.61:45951), d20a65bcd62d4aa7a0f120d0021ce8e3 (127.15.208.60:41567)
I20260619 08:55:22.197329 6137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3"
I20260619 08:55:22.197435 6137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:22.198380 6137 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ee6c5b9656e49dc8e0b435b0c8c561a in term 1.
I20260619 08:55:22.198601 5961 leader_election.cc:302] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ee6c5b9656e49dc8e0b435b0c8c561a, d20a65bcd62d4aa7a0f120d0021ce8e3; no voters:
I20260619 08:55:22.198733 6183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:22.198876 6183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [term 1 LEADER]: Becoming Leader. State: Replica: 1ee6c5b9656e49dc8e0b435b0c8c561a, State: Running, Role: LEADER
I20260619 08:55:22.198962 6183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } }
I20260619 08:55:22.199397 6183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ee6c5b9656e49dc8e0b435b0c8c561a. Latest consensus state: current_term: 1 leader_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } } }
I20260619 08:55:22.199496 6183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:22.200700 5944 master_runner.cc:186] Error getting master registration for 127.15.208.61:45951: OK, instance_id { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" instance_seqno: 1781859321983120 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:55:22.203073 5944 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 08:55:22.205384 6201 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:55:22.205430 6201 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:55:22.205549 6202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:22.205695 6202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:22.207245 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.198303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51966 (local address 127.15.208.61:45951)
0619 08:55:22.198345 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.198347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.198463 (+ 116us) server_negotiation.cc:407] Connection header received
0619 08:55:22.198615 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.198616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.198627 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.198658 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.199787 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.200543 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.201608 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.202373 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.202456 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.203984 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.203986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.203986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.203994 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.205763 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.206245 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.206247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.206247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.206309 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.206528 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.206529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.206529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.206665 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.206758 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.206922 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.207041 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:55:22.210582 6137 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [term 1 FOLLOWER]: Refusing update from remote peer 1ee6c5b9656e49dc8e0b435b0c8c561a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:55:22.212374 6199 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:55:22.214475 6169 sys_catalog.cc:455] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ee6c5b9656e49dc8e0b435b0c8c561a. Latest consensus state: current_term: 1 leader_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } } }
I20260619 08:55:22.214576 6169 sys_catalog.cc:458] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:22.216008 6015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.205076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56566 (local address 127.15.208.62:34891)
0619 08:55:22.205138 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.205140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.205865 (+ 725us) server_negotiation.cc:407] Connection header received
0619 08:55:22.206006 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.206007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.206029 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.206058 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.206832 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.207565 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.208568 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.208910 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.208972 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.210422 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.210423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.210424 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.210432 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.212257 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.214765 (+ 2508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.214766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.214767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.214848 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.215107 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.215108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.215109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.215264 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.215355 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.215853 (+ 498us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.215925 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:55:22.216008 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.205533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42466 (local address 127.15.208.60:41567)
0619 08:55:22.205575 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.205578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.207199 (+ 1621us) server_negotiation.cc:407] Connection header received
0619 08:55:22.207299 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.207300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.207312 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.207347 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.208396 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.209093 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.211348 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.211623 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.211686 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.213117 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.213119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.213119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.213127 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.214782 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.215182 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.215183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.215183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.215232 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.215433 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.215434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.215434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.215563 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.215627 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.215839 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.215917 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:55:22.220232 6087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } attempt: 3
I20260619 08:55:22.222103 6199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } } }
I20260619 08:55:22.222204 6199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:22.222424 6206 mvcc.cc:205] Tried to move back new op lower bound from 7298495783767445504 to 7298495783727579136. Current Snapshot: MvccSnapshot[applied={T|T < 7298495783767445504}]
I20260619 08:55:22.222738 6169 sys_catalog.cc:455] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } } }
I20260619 08:55:22.222819 6169 sys_catalog.cc:458] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:22.222895 6207 mvcc.cc:205] Tried to move back new op lower bound from 7298495783767445504 to 7298495783727579136. Current Snapshot: MvccSnapshot[applied={T|T < 7298495783767445504}]
Jun 19 08:55:22 dist-test-slave-p0h9 krb5kdc[5931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
I20260619 08:55:22.229256 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.220502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:41567 (local address 127.0.0.1:42480)
0619 08:55:22.220549 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:22.220560 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:22.220606 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:22.220772 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:22.220772 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:22.220981 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:22.221034 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.221036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.222351 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.222352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.223174 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:22.223177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.223683 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.223684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:22.223758 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.224228 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:22.224233 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:22.226021 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:22.228094 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.228095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.228095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.228320 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.228546 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:22.228546 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:22.228547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:22.228617 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:22.228912 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:22.228913 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:22.228970 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 08:55:22.229032 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 08:55:22.229138 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 08:55:22.229382 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:22.220537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.15.208.60:41567)
0619 08:55:22.220568 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.220570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.220590 (+ 20us) server_negotiation.cc:407] Connection header received
0619 08:55:22.220666 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.220667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.220676 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.221549 (+ 873us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.221577 (+ 28us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.222247 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.223283 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:22.223591 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:22.223655 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:22.226118 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:22.226119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:22.226119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:22.226125 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:22.227982 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.228401 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.228402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.228402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.228466 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:22.228679 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:22.228680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:22.228680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:22.228846 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:22.228907 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:22.229062 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 08:55:22.229239 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:55:22.231141 6087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5: Bootstrap starting.
I20260619 08:55:22.231858 6087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:22.232148 6087 log.cc:824] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:22.233175 6202 catalog_manager.cc:1347] Generated new cluster ID: 233a6e1aa60c4b42a3f9c0495d29442d
I20260619 08:55:22.233246 6202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:22.241377 6087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5: No bootstrap required, opened a new log
I20260619 08:55:22.243242 6087 raft_consensus.cc:348] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } }
I20260619 08:55:22.243413 6087 raft_consensus.cc:729] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adbf4a7c389a4c6db33babd7445a96a5, State: Initialized, Role: FOLLOWER
I20260619 08:55:22.243479 6087 consensus_queue.cc:260] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } }
I20260619 08:55:22.243778 6087 sys_catalog.cc:564] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:22.243894 6060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:22.244837 6061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "adbf4a7c389a4c6db33babd7445a96a5"
I20260619 08:55:22.245177 6060 raft_consensus.cc:1261] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [term 1 FOLLOWER]: Refusing update from remote peer 1ee6c5b9656e49dc8e0b435b0c8c561a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:55:22.245498 6183 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [LEADER]: Connected to new peer: Peer: permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:55:22.246282 6061 raft_consensus.cc:2454] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ee6c5b9656e49dc8e0b435b0c8c561a in term 1.
I20260619 08:55:22.246425 6212 sys_catalog.cc:455] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } } }
I20260619 08:55:22.247058 6212 sys_catalog.cc:458] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:22.247227 6212 sys_catalog.cc:455] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ee6c5b9656e49dc8e0b435b0c8c561a. Latest consensus state: current_term: 1 leader_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } } }
I20260619 08:55:22.247344 6212 sys_catalog.cc:458] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:22.247416 6212 sys_catalog.cc:455] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } } }
I20260619 08:55:22.247503 6212 sys_catalog.cc:458] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:22.248857 6202 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:22.249576 6202 catalog_manager.cc:1504] Loading token signing keys...
W20260619 08:55:22.253247 6223 catalog_manager.cc:1558] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:55:22.253371 6223 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:55:22.256393 6225 mvcc.cc:205] Tried to move back new op lower bound from 7298495783767445504 to 7298495783727579136. Current Snapshot: MvccSnapshot[applied={T|T < 7298495783767445504}]
I20260619 08:55:22.257254 6199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee6c5b9656e49dc8e0b435b0c8c561a" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 34891 } } peers { permanent_uuid: "adbf4a7c389a4c6db33babd7445a96a5" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 45951 } } peers { permanent_uuid: "d20a65bcd62d4aa7a0f120d0021ce8e3" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 41567 } } }
I20260619 08:55:22.257339 6199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:22.265136 6202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ee6c5b9656e49dc8e0b435b0c8c561a: Generated new TSK 0
I20260619 08:55:22.265313 6202 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:55:22.293996 6015 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.292378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56576 (local address 127.15.208.62:34891)
0619 08:55:22.292438 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.292440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.293311 (+ 871us) server_negotiation.cc:407] Connection header received
0619 08:55:22.293421 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.293423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.293436 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.293467 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.293926 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56576: BlockingRecv error: recv got EOF from 127.0.0.1:56576 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 08:55:22.294742 6167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.292629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42492 (local address 127.15.208.60:41567)
0619 08:55:22.292672 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.292674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.294221 (+ 1547us) server_negotiation.cc:407] Connection header received
0619 08:55:22.294316 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.294317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.294327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.294355 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.294684 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42492: BlockingRecv error: recv got EOF from 127.0.0.1:42492 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 08:55:22.295429 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.293028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51982 (local address 127.15.208.61:45951)
0619 08:55:22.293069 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.293070 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.294951 (+ 1881us) server_negotiation.cc:407] Connection header received
0619 08:55:22.295045 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.295046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.295057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.295086 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.295370 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51982: BlockingRecv error: recv got EOF from 127.0.0.1:51982 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:55:22.297730 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.296949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51994 (local address 127.15.208.61:45951)
0619 08:55:22.296986 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.296988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.297218 (+ 230us) server_negotiation.cc:407] Connection header received
0619 08:55:22.297324 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.297325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.297341 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.297370 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.297687 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51994: BlockingRecv error: recv got EOF from 127.0.0.1:51994 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 08:55:22.300561 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.299897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52008 (local address 127.15.208.61:45951)
0619 08:55:22.299929 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.299930 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.300126 (+ 196us) server_negotiation.cc:407] Connection header received
0619 08:55:22.300213 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.300214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.300223 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.300249 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.300523 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52008: BlockingRecv error: recv got EOF from 127.0.0.1:52008 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 08:55:22.304344 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.303670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52012 (local address 127.15.208.61:45951)
0619 08:55:22.303700 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.303702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.303905 (+ 203us) server_negotiation.cc:407] Connection header received
0619 08:55:22.303989 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.303990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.304000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.304026 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.304305 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52012: BlockingRecv error: recv got EOF from 127.0.0.1:52012 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 08:55:22.310235 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.309475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52018 (local address 127.15.208.61:45951)
0619 08:55:22.309512 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.309514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.309746 (+ 232us) server_negotiation.cc:407] Connection header received
0619 08:55:22.309842 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.309843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.309853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.309880 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.310193 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52018: BlockingRecv error: recv got EOF from 127.0.0.1:52018 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 08:55:22.320223 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.319424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52022 (local address 127.15.208.61:45951)
0619 08:55:22.319460 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.319462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.319696 (+ 234us) server_negotiation.cc:407] Connection header received
0619 08:55:22.319794 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.319795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.319807 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.319836 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.320175 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52022: BlockingRecv error: recv got EOF from 127.0.0.1:52022 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 08:55:22.338817 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.337889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.15.208.61:45951)
0619 08:55:22.337945 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.337947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.338202 (+ 255us) server_negotiation.cc:407] Connection header received
0619 08:55:22.338308 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.338309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.338322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.338352 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.338760 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52024: BlockingRecv error: recv got EOF from 127.0.0.1:52024 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 08:55:22.375166 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.373796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.15.208.61:45951)
0619 08:55:22.373838 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.373840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.374524 (+ 684us) server_negotiation.cc:407] Connection header received
0619 08:55:22.374646 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.374647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.374661 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.374692 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.375089 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52032: BlockingRecv error: recv got EOF from 127.0.0.1:52032 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:55:22.441431 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.440801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.15.208.61:45951)
0619 08:55:22.440869 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.440872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.440883 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:55:22.440954 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.440956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.440967 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.441019 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.441359 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52048: BlockingRecv error: recv got EOF from 127.0.0.1:52048 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 08:55:22.571410 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.570608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52062 (local address 127.15.208.61:45951)
0619 08:55:22.570701 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.570703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.570808 (+ 105us) server_negotiation.cc:407] Connection header received
0619 08:55:22.570907 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.570908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.570921 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.570999 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.571344 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52062: BlockingRecv error: recv got EOF from 127.0.0.1:52062 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 08:55:22.829466 6091 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:22.828778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52072 (local address 127.15.208.61:45951)
0619 08:55:22.828856 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:22.828858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:22.828870 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:55:22.828951 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:22.828952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:22.828964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:22.829039 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:22.829419 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52072: BlockingRecv error: recv got EOF from 127.0.0.1:52072 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:55:23.187513 6180 catalog_manager.cc:1259] Loaded cluster ID: 233a6e1aa60c4b42a3f9c0495d29442d
I20260619 08:55:23.187590 6180 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3: loading cluster ID for follower catalog manager: success
I20260619 08:55:23.188422 6180 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3: acquiring CA information for follower catalog manager: success
I20260619 08:55:23.188763 6180 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d20a65bcd62d4aa7a0f120d0021ce8e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:55:23.253924 6223 catalog_manager.cc:1259] Loaded cluster ID: 233a6e1aa60c4b42a3f9c0495d29442d
I20260619 08:55:23.254005 6223 catalog_manager.cc:1552] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5: loading cluster ID for follower catalog manager: success
I20260619 08:55:23.254827 6223 catalog_manager.cc:1574] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5: acquiring CA information for follower catalog manager: success
I20260619 08:55:23.255131 6223 catalog_manager.cc:1602] T 00000000000000000000000000000000 P adbf4a7c389a4c6db33babd7445a96a5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:55:23.346077 6294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:23.342779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52088 (local address 127.15.208.61:45951)
0619 08:55:23.342930 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:23.342933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:23.342989 (+ 56us) server_negotiation.cc:407] Connection header received
0619 08:55:23.343081 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:23.343082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:23.343156 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:23.343258 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:55:23.343470 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:23.344124 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:23.345162 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:23.345375 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:23.345453 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:23.345679 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:55:23.345792 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:55:23.345887 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:23.345931 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:55:23.345955 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":58,"threads_started":1}
I20260619 08:55:23.350647 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:23.347968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42504 (local address 127.15.208.60:41567)
0619 08:55:23.348097 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:23.348099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:23.348111 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:55:23.348171 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:23.348172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:23.348218 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:23.348312 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:55:23.348415 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:23.348958 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:23.349779 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:23.349994 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:23.350086 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:23.350265 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:55:23.350377 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:55:23.350474 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:23.350518 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:55:23.350542 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":52,"threads_started":1}
I20260619 08:55:23.355155 6294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:23.352445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52096 (local address 127.15.208.61:45951)
0619 08:55:23.352502 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:23.352504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:23.352759 (+ 255us) server_negotiation.cc:407] Connection header received
0619 08:55:23.352877 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:23.352878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:23.352890 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:23.352961 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:55:23.353127 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:23.353685 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:23.354417 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:23.354608 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:23.354703 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:23.354858 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:55:23.354895 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:55:23.354954 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:23.355039 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 08:55:23.355059 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 08:55:23.355867 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:23.352708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.15.208.60:41567)
0619 08:55:23.352771 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:23.352773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:23.352959 (+ 186us) server_negotiation.cc:407] Connection header received
0619 08:55:23.353002 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:23.353003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:23.353014 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:23.353062 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:55:23.353209 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:23.353905 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:23.355055 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:23.355341 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:23.355386 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:23.355618 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:55:23.355662 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:55:23.355717 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:23.355773 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 08:55:23.355796 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:55:23.356087 6307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:23.352518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56578 (local address 127.15.208.62:34891)
0619 08:55:23.352620 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:23.352622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:23.352631 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:55:23.352706 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:23.352707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:23.352785 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:23.352873 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:55:23.353034 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:23.353876 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:23.355055 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:23.355356 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:23.355420 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:23.355651 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:55:23.355764 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:55:23.355827 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:23.355912 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 08:55:23.356032 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
I20260619 08:55:23.357895 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 5944
I20260619 08:55:23.363094 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 6019
I20260619 08:55:23.368113 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 6094
2026-06-19T08:55:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2795 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:55:23 dist-test-slave-p0h9 krb5kdc[6311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:55:23 dist-test-slave-p0h9 krb5kdc[6311](info): set up 2 sockets
Jun 19 08:55:23 dist-test-slave-p0h9 krb5kdc[6311](info): commencing operation
krb5kdc: starting...
W20260619 08:55:24.409682 16192 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/test-user.keytab.
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:55:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:55:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
Entry for principal kudu/127.15.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab.
I20260619 08:55:24.525470 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:43689
--webserver_interface=127.15.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:43689,127.15.208.61:44041,127.15.208.60:45397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:55:24.605549 6324 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:55:24.605796 6324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:24.605829 6324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:24.605870 6324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:24.606986 6324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:24.607059 6324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:24.607081 6324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:24.607101 6324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:24.607131 6324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:24.608618 6324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:43689,127.15.208.61:44041,127.15.208.60:45397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.62.keytab
--principal=kudu/127.15.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.62:43689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:24.608920 6324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:24.609162 6324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:24.614078 6330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:24.614084 6332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:24.614157 6324 server_base.cc:1034] running on GCE node
W20260619 08:55:24.614095 6329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:24.614481 6324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:24.614806 6324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:24.615971 6324 hybrid_clock.cc:648] HybridClock initialized: now 1781859324615953 us; error 42 us; skew 500 ppm
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:24.618439 6324 init.cc:377] Logged in from keytab as kudu/127.15.208.62@KRBTEST.COM (short username kudu)
I20260619 08:55:24.618860 6324 webserver.cc:460] Webserver started at http://127.15.208.62:33765/ using document root <none> and password file <none>
I20260619 08:55:24.619109 6324 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:24.619180 6324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:24.619323 6324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:24.620368 6324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/data/instance:
uuid: "1277ef8abbfe48959c901b93ed0993d9"
format_stamp: "Formatted at 2026-06-19 08:55:24 on dist-test-slave-p0h9"
server_key: "5cc0a8e03c878b18143c3485b920eac3"
server_key_iv: "8bf71819f084545ba810e42c37f55306"
server_key_version: "encryptionkey@0"
I20260619 08:55:24.620716 6324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/wal/instance:
uuid: "1277ef8abbfe48959c901b93ed0993d9"
format_stamp: "Formatted at 2026-06-19 08:55:24 on dist-test-slave-p0h9"
server_key: "5cc0a8e03c878b18143c3485b920eac3"
server_key_iv: "8bf71819f084545ba810e42c37f55306"
server_key_version: "encryptionkey@0"
I20260619 08:55:24.622054 6324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:55:24.622733 6339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:24.622903 6324 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:55:24.623001 6324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/wal
uuid: "1277ef8abbfe48959c901b93ed0993d9"
format_stamp: "Formatted at 2026-06-19 08:55:24 on dist-test-slave-p0h9"
server_key: "5cc0a8e03c878b18143c3485b920eac3"
server_key_iv: "8bf71819f084545ba810e42c37f55306"
server_key_version: "encryptionkey@0"
I20260619 08:55:24.623081 6324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:24.664443 6324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:24.666580 6324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:24.666751 6324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:24.670745 6324 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.62:43689
I20260619 08:55:24.670819 6391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.62:43689 every 8 connection(s)
I20260619 08:55:24.671188 6324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/data/info.pb
I20260619 08:55:24.672204 6392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:24.673801 6392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:55:24.680624 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 6324
I20260619 08:55:24.680751 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-0/wal/instance
I20260619 08:55:24.680987 16192 external_mini_cluster.cc:1428] Setting key 76ea82ca16ada1323e161eaf930ac0e9
I20260619 08:55:24.683925 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.674574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55878 (local address 127.15.208.62:43689)
0619 08:55:24.674756 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:24.674766 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:24.674892 (+ 126us) server_negotiation.cc:407] Connection header received
0619 08:55:24.675345 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:24.675347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:24.675363 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:24.675443 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:24.676048 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.676929 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.677787 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.678079 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.678165 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.680142 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:24.680149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:24.680149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:24.680155 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:24.682204 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.682682 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.682685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.682685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.682765 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.683033 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.683034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.683034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.683242 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:24.683319 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:24.683566 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 08:55:24.683746 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
I20260619 08:55:24.683998 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.674558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43689 (local address 127.0.0.1:55878)
0619 08:55:24.674740 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.674780 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:24.675225 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:24.675464 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:24.675467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:24.675737 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:24.675909 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.675913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.677010 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.677011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.677718 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.677719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.678168 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.678169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.678221 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.678726 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:24.678734 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:24.680065 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:24.682288 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.682295 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.682297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.682568 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.682835 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.682836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.682836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.682965 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.683326 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:24.683333 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:24.683409 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 08:55:24.683527 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 08:55:24.683810 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":74,"thread_start_us":111,"threads_started":1}
I20260619 08:55:24.684772 6392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:24.685181 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.685029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:44041 (local address 127.0.0.1:43050)
0619 08:55:24.685087 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.685144 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 08:55:24.685329 6340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.208.61:44041: connect: Connection refused (error 111)
I20260619 08:55:24.685827 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.685711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:44041 (local address 127.0.0.1:43056)
0619 08:55:24.685761 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.685794 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 08:55:24.685966 6392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.61:44041: Network error: Client connection negotiation failed: client connection to 127.15.208.61:44041: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.15.208.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab.
Entry for principal kudu/127.15.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab.
I20260619 08:55:24.707158 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.61:44041
--webserver_interface=127.15.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab
--principal=kudu/127.15.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:43689,127.15.208.61:44041,127.15.208.60:45397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:55:24.735316 6392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } attempt: 1
I20260619 08:55:24.735920 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.735755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:44041 (local address 127.0.0.1:43060)
0619 08:55:24.735826 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.735878 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 08:55:24.736300 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.736205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:44041 (local address 127.0.0.1:43062)
0619 08:55:24.736249 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.736278 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:55:24.736420 6392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.61:44041: Network error: Client connection negotiation failed: client connection to 127.15.208.61:44041: connect: Connection refused (error 111)
W20260619 08:55:24.788986 6399 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:55:24.789201 6399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:24.789218 6399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:24.789261 6399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:24.790403 6399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:24.790452 6399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:24.790467 6399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:24.790480 6399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:24.790493 6399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:24.791980 6399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:43689,127.15.208.61:44041,127.15.208.60:45397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.61.keytab
--principal=kudu/127.15.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.61:44041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:24.792227 6399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:24.792456 6399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:55:24.797251 6405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:24.797264 6404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:24.797277 6407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:24.797473 6399 server_base.cc:1034] running on GCE node
I20260619 08:55:24.797710 6399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:24.797997 6399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:24.799168 6399 hybrid_clock.cc:648] HybridClock initialized: now 1781859324799160 us; error 38 us; skew 500 ppm
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:24.801645 6399 init.cc:377] Logged in from keytab as kudu/127.15.208.61@KRBTEST.COM (short username kudu)
I20260619 08:55:24.802091 6399 webserver.cc:460] Webserver started at http://127.15.208.61:38773/ using document root <none> and password file <none>
I20260619 08:55:24.802345 6399 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:24.802412 6399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:24.802560 6399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:24.803617 6399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/data/instance:
uuid: "93a3f8a91fb04b21badb68c7c3b4ad85"
format_stamp: "Formatted at 2026-06-19 08:55:24 on dist-test-slave-p0h9"
server_key: "81d955d3e7dd01ec9893e0f71c28276e"
server_key_iv: "e1bb2c1159f54139245a0b7d34af5a93"
server_key_version: "encryptionkey@0"
I20260619 08:55:24.803975 6399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/wal/instance:
uuid: "93a3f8a91fb04b21badb68c7c3b4ad85"
format_stamp: "Formatted at 2026-06-19 08:55:24 on dist-test-slave-p0h9"
server_key: "81d955d3e7dd01ec9893e0f71c28276e"
server_key_iv: "e1bb2c1159f54139245a0b7d34af5a93"
server_key_version: "encryptionkey@0"
I20260619 08:55:24.804615 6392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } attempt: 2
I20260619 08:55:24.805011 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.804890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:44041 (local address 127.0.0.1:43078)
0619 08:55:24.804949 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.804982 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:55:24.805466 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.805343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:44041 (local address 127.0.0.1:43094)
0619 08:55:24.805399 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.805431 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.61:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 08:55:24.805572 6392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.61:44041: Network error: Client connection negotiation failed: client connection to 127.15.208.61:44041: connect: Connection refused (error 111)
I20260619 08:55:24.805630 6399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:55:24.806396 6414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:24.806576 6399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:55:24.806674 6399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/wal
uuid: "93a3f8a91fb04b21badb68c7c3b4ad85"
format_stamp: "Formatted at 2026-06-19 08:55:24 on dist-test-slave-p0h9"
server_key: "81d955d3e7dd01ec9893e0f71c28276e"
server_key_iv: "e1bb2c1159f54139245a0b7d34af5a93"
server_key_version: "encryptionkey@0"
I20260619 08:55:24.806761 6399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:24.821583 6399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:24.823418 6399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:24.823589 6399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:24.827806 6399 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.61:44041
I20260619 08:55:24.827838 6466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.61:44041 every 8 connection(s)
I20260619 08:55:24.828202 6399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/data/info.pb
I20260619 08:55:24.829298 6467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:24.830776 6467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:24.831897 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 6399
I20260619 08:55:24.831977 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-1/wal/instance
I20260619 08:55:24.832161 16192 external_mini_cluster.cc:1428] Setting key abf37ff9cdf72bc6b2b9cadd36020d44
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.208.60@KRBTEST.COM; defaulting to no policy
I20260619 08:55:24.840194 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.831419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55882 (local address 127.15.208.62:43689)
0619 08:55:24.831462 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:24.831464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:24.831721 (+ 257us) server_negotiation.cc:407] Connection header received
0619 08:55:24.832343 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:24.832345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:24.832362 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:24.832416 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:24.833120 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.833658 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.834733 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.834949 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.835041 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.837303 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:24.837305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:24.837305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:24.837313 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:24.838728 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.839185 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.839187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.839187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.839250 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.839500 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.839501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.839502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.839617 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:24.839690 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:24.840010 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 08:55:24.840094 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:55:24.840361 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.831384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43689 (local address 127.0.0.1:55882)
0619 08:55:24.831561 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.831603 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:24.832160 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:24.832485 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:24.832489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:24.832816 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:24.833022 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.833027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.833767 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.833768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.834621 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.834623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.835069 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.835071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.835193 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.835807 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:24.835815 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:24.837192 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:24.838826 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.838833 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.838835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.839109 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.839318 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.839319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.839319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.839431 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.839696 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:24.839702 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:24.839858 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 08:55:24.839988 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 08:55:24.840137 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260619 08:55:24.840979 6467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.208.60@KRBTEST.COM" created.
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
I20260619 08:55:24.849510 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.841255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:44041 (local address 127.0.0.1:43108)
0619 08:55:24.841308 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.841334 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:24.841380 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:24.841710 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:24.841711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:24.841818 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:24.841863 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.841864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.843100 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.843101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.843880 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.843883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.844564 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.844565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.844616 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.845057 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:24.845061 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:24.846460 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:24.848461 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.848462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.848463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.848653 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.848917 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.848918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.848918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.848972 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.849286 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:24.849286 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:24.849321 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 08:55:24.849371 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:55:24.849448 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 08:55:24.849627 6471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.841367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.15.208.61:44041)
0619 08:55:24.841528 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:24.841533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:24.841557 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:55:24.841598 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:24.841601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:24.841617 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:24.841698 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:24.841938 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.843011 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.843964 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.844484 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.844564 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.846547 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:24.846555 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:24.846556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:24.846563 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:24.848354 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.848766 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.848769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.848770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.848833 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.849037 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.849037 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.849038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.849217 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:24.849301 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:24.849410 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 08:55:24.849518 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260619 08:55:24.850188 6467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:24.850559 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.850462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60842)
0619 08:55:24.850500 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.850542 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 08:55:24.850704 6417 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
I20260619 08:55:24.851079 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.851021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60850)
0619 08:55:24.851042 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.851065 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260619 08:55:24.851205 6467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:45397: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab.
Entry for principal kudu/127.15.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab.
I20260619 08:55:24.856921 16192 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLqpuKl/build/release/bin/kudu
/tmp/dist-test-taskLqpuKl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.60:45397
--webserver_interface=127.15.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab
--principal=kudu/127.15.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.208.20:35047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.208.62:43689,127.15.208.61:44041,127.15.208.60:45397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:55:24.896785 6392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } attempt: 3
I20260619 08:55:24.900470 6467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } attempt: 1
I20260619 08:55:24.900987 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.900842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60866)
0619 08:55:24.900894 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.900932 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 08:55:24.901418 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.901318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60876)
0619 08:55:24.901364 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.901390 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
W20260619 08:55:24.901563 6467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:45397: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
I20260619 08:55:24.904482 6471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.897281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43120 (local address 127.15.208.61:44041)
0619 08:55:24.897342 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:24.897344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:24.897363 (+ 19us) server_negotiation.cc:407] Connection header received
0619 08:55:24.897390 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:24.897390 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:24.897400 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:24.897452 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:24.897845 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.898333 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.899371 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.899580 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.899635 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.901898 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:24.901900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:24.901900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:24.901908 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:24.903347 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.903706 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.903707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.903708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.903767 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.903965 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.903965 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.903966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.904080 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:24.904151 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:24.904318 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 08:55:24.904395 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:55:24.904488 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.897177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:44041 (local address 127.0.0.1:43120)
0619 08:55:24.897252 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.897268 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:24.897334 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:24.897478 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:24.897479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:24.897676 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:24.897765 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.897768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.898415 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.898417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.899282 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.899285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.899652 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.899653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.899732 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.900211 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:24.900217 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:24.901808 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:24.903450 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.903452 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.903452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.903628 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.903834 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.903835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.903835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.903892 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.904164 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:24.904165 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:24.904212 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 08:55:24.904294 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:55:24.904394 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 08:55:24.904943 6392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:24.905310 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.905216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60878)
0619 08:55:24.905264 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.905292 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:55:24.905675 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.905587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60890)
0619 08:55:24.905630 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.905653 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 08:55:24.905769 6392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:45397: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
I20260619 08:55:24.936928 6392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } attempt: 1
I20260619 08:55:24.937393 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.937254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60892)
0619 08:55:24.937326 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.937365 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 08:55:24.937788 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.937697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60894)
0619 08:55:24.937748 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.937773 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 08:55:24.937887 6392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:45397: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
W20260619 08:55:24.940088 6474 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:55:24.940280 6474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:55:24.940325 6474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:55:24.940366 6474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:55:24.941591 6474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:55:24.941638 6474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:55:24.941653 6474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:55:24.941664 6474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:55:24.941712 6474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:55:24.943109 6474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.208.20:35047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.208.62:43689,127.15.208.61:44041,127.15.208.60:45397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/krb5kdc/kudu_127.15.208.60.keytab
--principal=kudu/127.15.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.208.60:45397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 08:01:05 UTC on ccycloud.quasar-zjmztc.root.comops.site
build id 4455298
I20260619 08:55:24.943393 6474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:55:24.943645 6474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:55:24.948364 6474 server_base.cc:1034] running on GCE node
W20260619 08:55:24.948339 6482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:24.948321 6479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:55:24.948318 6480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:55:24.948818 6474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:55:24.949172 6474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:55:24.950340 6474 hybrid_clock.cc:648] HybridClock initialized: now 1781859324950333 us; error 35 us; skew 500 ppm
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:55:24.952682 6474 init.cc:377] Logged in from keytab as kudu/127.15.208.60@KRBTEST.COM (short username kudu)
I20260619 08:55:24.953117 6474 webserver.cc:460] Webserver started at http://127.15.208.60:42697/ using document root <none> and password file <none>
I20260619 08:55:24.953342 6474 fs_manager.cc:359] Metadata directory not provided
I20260619 08:55:24.953408 6474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:55:24.953555 6474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:55:24.954598 6474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/data/instance:
uuid: "1b616ee16391415fb425e9b6a9a57e47"
format_stamp: "Formatted at 2026-06-19 08:55:24 on dist-test-slave-p0h9"
server_key: "0e1c99f814fa5683a2694efca15e9363"
server_key_iv: "e317f5fe87906ae8c3af464333186131"
server_key_version: "encryptionkey@0"
I20260619 08:55:24.954957 6474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/wal/instance:
uuid: "1b616ee16391415fb425e9b6a9a57e47"
format_stamp: "Formatted at 2026-06-19 08:55:24 on dist-test-slave-p0h9"
server_key: "0e1c99f814fa5683a2694efca15e9363"
server_key_iv: "e317f5fe87906ae8c3af464333186131"
server_key_version: "encryptionkey@0"
I20260619 08:55:24.956339 6474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:55:24.957175 6489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:55:24.957366 6474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:55:24.957466 6474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/data,/tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/wal
uuid: "1b616ee16391415fb425e9b6a9a57e47"
format_stamp: "Formatted at 2026-06-19 08:55:24 on dist-test-slave-p0h9"
server_key: "0e1c99f814fa5683a2694efca15e9363"
server_key_iv: "e317f5fe87906ae8c3af464333186131"
server_key_version: "encryptionkey@0"
I20260619 08:55:24.957541 6474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:55:24.969583 6474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:55:24.969755 6467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } attempt: 2
I20260619 08:55:24.970222 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.970090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60896)
0619 08:55:24.970148 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.970180 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 08:55:24.970665 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.970544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60906)
0619 08:55:24.970622 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.970648 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260619 08:55:24.971052 6467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.208.60:45397: Network error: Client connection negotiation failed: client connection to 127.15.208.60:45397: connect: Connection refused (error 111)
I20260619 08:55:24.972113 6474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:55:24.972263 6474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:55:24.976264 6474 rpc_server.cc:307] RPC server started. Bound to: 127.15.208.60:45397
I20260619 08:55:24.976295 6542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.208.60:45397 every 8 connection(s)
I20260619 08:55:24.976652 6474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/data/info.pb
I20260619 08:55:24.977566 6543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:55:24.979115 6543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:24.981413 16192 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLqpuKl/build/release/bin/kudu as pid 6474
I20260619 08:55:24.981519 16192 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLqpuKl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781859057751076-16192-0/minicluster-data/master-2/wal/instance
I20260619 08:55:24.981726 16192 external_mini_cluster.cc:1428] Setting key 2436b3d23ed07ca9884364d68b74b949
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
I20260619 08:55:24.988580 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.979810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.15.208.62:43689)
0619 08:55:24.979870 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:24.979872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:24.980054 (+ 182us) server_negotiation.cc:407] Connection header received
0619 08:55:24.980491 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:24.980492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:24.980500 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:24.980564 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:24.981182 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.981932 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.983118 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.983315 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.983367 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.985678 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:24.985679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:24.985679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:24.985686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:24.987117 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.987602 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.987603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.987611 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.987673 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.987907 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.987908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.987909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.988028 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:24.988106 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:24.988387 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 08:55:24.988473 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:55:24.988735 6545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.979776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.62:43689 (local address 127.0.0.1:55896)
0619 08:55:24.979932 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.979972 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:24.980413 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:24.980620 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:24.980624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:24.980914 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:24.981102 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.981113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.982038 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.982039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.983031 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.983033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.983422 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.983423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.983551 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.984007 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:24.984017 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:24.985565 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:24.987201 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.987207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.987209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.987526 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.987743 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.987743 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.987744 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.987835 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.988097 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:24.988103 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:24.988228 (+ 125us) client_negotiation.cc:769] Sending connection context
0619 08:55:24.988332 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 08:55:24.988502 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260619 08:55:24.989588 6543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:24.991058 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.982082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.15.208.60:45397)
0619 08:55:24.982260 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:24.982265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:24.982319 (+ 54us) server_negotiation.cc:407] Connection header received
0619 08:55:24.982369 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:24.982372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:24.982391 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:24.982461 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:24.983028 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.984011 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.984715 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.985219 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.985283 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.987877 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:24.987884 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:24.987884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:24.987891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:24.989851 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.990240 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.990243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.990243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.990327 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.990517 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.990517 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.990517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.990684 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:24.990808 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:24.990872 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 08:55:24.990943 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.62@KRBTEST.COM
I20260619 08:55:24.997133 6471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.989946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43136 (local address 127.15.208.61:44041)
0619 08:55:24.990009 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:24.990012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:24.990020 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:55:24.990044 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:24.990045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:24.990055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:24.990116 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:24.990343 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.991061 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.992040 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.992393 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.992468 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.994379 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:24.994381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:24.994381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:24.994389 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:24.995987 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.996389 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.996390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.996390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.996439 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.996642 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.996643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.996643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.996790 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:24.996859 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:24.997006 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 08:55:24.997071 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:55:24.997175 6545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.989889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.61:44041 (local address 127.0.0.1:43136)
0619 08:55:24.989929 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.989939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:24.989975 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:24.990127 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:24.990127 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:24.990237 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:24.990265 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.990266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.991239 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.991240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.991916 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.991920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.992472 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.992473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.992516 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.992925 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:24.992929 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:24.994292 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:24.996078 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.996079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.996079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.996316 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.996507 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:24.996508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:24.996508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:24.996574 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:24.996861 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:24.996862 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:24.996906 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 08:55:24.996979 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:55:24.997075 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 08:55:24.997481 6543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:55:24.999984 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.991830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55900 (local address 127.15.208.62:43689)
0619 08:55:24.991887 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:24.991889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:24.991920 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:55:24.991979 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:24.991980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:24.991991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:24.992043 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:24.992638 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.993380 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.994381 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.994611 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.994678 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:24.997326 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:24.997327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:24.997327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:24.997335 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:24.998860 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.999207 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.999209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.999209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.999261 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:24.999469 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:24.999470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:24.999470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:24.999589 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:24.999777 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:24.999814 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 08:55:24.999881 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 08:55:24 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.60@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
I20260619 08:55:25.003921 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.997787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60920 (local address 127.15.208.60:45397)
0619 08:55:24.997828 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:24.997830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:24.997838 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:55:24.997859 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:24.997860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:24.997871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:24.997937 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:24.998146 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.998622 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.999243 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.999445 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.999527 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:25.001506 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:25.001507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:25.001508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:25.001514 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:25.002843 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.003184 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.003185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.003186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.003239 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.003473 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.003474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.003475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.003583 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:25.003636 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:25.003803 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 08:55:25.003864 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:55:25.004055 6545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:24.997731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60920)
0619 08:55:24.997764 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:24.997774 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:24.997815 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:24.997938 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:24.997939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:24.998048 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:24.998074 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.998075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.998704 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.998705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.999155 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:24.999156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:24.999539 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:24.999540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:24.999608 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:25.000003 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:25.000008 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:25.001445 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:25.002944 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:25.002945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:25.002945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:25.003103 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:25.003318 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:25.003319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:25.003319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:25.003400 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:25.003652 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:25.003653 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:25.003701 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 08:55:25.003789 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:55:25.003934 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":44}
Jun 19 08:55:25 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.208.61@KRBTEST.COM
I20260619 08:55:25.005720 6543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47: Bootstrap starting.
I20260619 08:55:25.006850 6543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:25.007176 6543 log.cc:824] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:25.008001 6543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47: No bootstrap required, opened a new log
I20260619 08:55:25.008958 6471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:25.000652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43142 (local address 127.15.208.61:44041)
0619 08:55:25.000686 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.000688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.000696 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:55:25.000811 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.000812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.000820 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.000862 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.001426 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.002023 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.002755 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.002970 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.003039 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:25.005659 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:25.005660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:25.005660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:25.005667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:25.007833 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.008212 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.008213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.008214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.008272 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.008453 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.008453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.008454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.008580 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:25.008764 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:25.008802 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 08:55:25.008878 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:55:25.010054 6543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } }
I20260619 08:55:25.010174 6543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b616ee16391415fb425e9b6a9a57e47, State: Initialized, Role: FOLLOWER
I20260619 08:55:25.010262 6543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } }
I20260619 08:55:25.010485 16192 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:55:25.010617 6543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:25.010596 6549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } } }
I20260619 08:55:25.010679 6549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:55:25.012334 6560 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:55:25.012384 6560 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:55:25.011498 6366 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1277ef8abbfe48959c901b93ed0993d9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55900
I20260619 08:55:25.013062 6392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } attempt: 2
I20260619 08:55:25.012635 6474 master_runner.cc:186] Error getting master registration for 127.15.208.62:43689: OK, instance_id { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" instance_seqno: 1781859324669652 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:55:25.015064 6474 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 19 08:55:25 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.62@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
I20260619 08:55:25.021556 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:25.013297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60922)
0619 08:55:25.013342 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:25.013352 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:25.013387 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:25.013587 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:25.013588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:25.013783 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:25.013842 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:25.013844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.015204 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.015206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:25.015997 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:25.016000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.016484 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.016485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:25.016543 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:25.017056 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:25.017061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:25.018655 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:25.020418 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:25.020419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:25.020419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:25.020643 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:25.020903 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:25.020904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:25.020905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:25.020986 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:25.021235 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:25.021236 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:25.021297 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 08:55:25.021360 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 08:55:25.021459 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 08:55:25.021600 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:25.013364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.15.208.60:45397)
0619 08:55:25.013412 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.013414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.013423 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:55:25.013481 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.013482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.013491 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.013584 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.013965 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.014977 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.016090 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.016407 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.016471 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:25.018754 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:25.018755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:25.018755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:25.018762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:25.020342 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.020786 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.020787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.020788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.020849 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.021057 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.021057 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.021058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.021187 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:25.021235 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:25.021388 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 08:55:25.021505 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:55:25.023468 6392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9: Bootstrap starting.
I20260619 08:55:25.024204 6392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:25.024497 6392 log.cc:824] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:25.025244 6392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9: No bootstrap required, opened a new log
I20260619 08:55:25.026607 6392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } }
I20260619 08:55:25.026747 6392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1277ef8abbfe48959c901b93ed0993d9, State: Initialized, Role: FOLLOWER
I20260619 08:55:25.026832 6392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } }
I20260619 08:55:25.027189 6392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:25.027166 6563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } } }
I20260619 08:55:25.027262 6366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 08:55:25.027361 6563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:25.027423 6366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:25.028343 6366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } }
I20260619 08:55:25.029057 6517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1277ef8abbfe48959c901b93ed0993d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1b616ee16391415fb425e9b6a9a57e47"
I20260619 08:55:25.029199 6517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:25.028635 6366 leader_election.cc:288] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [CANDIDATE]: Term 1 election: Requested vote from peers 93a3f8a91fb04b21badb68c7c3b4ad85 (127.15.208.61:44041), 1b616ee16391415fb425e9b6a9a57e47 (127.15.208.60:45397)
I20260619 08:55:25.030081 6517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1277ef8abbfe48959c901b93ed0993d9 in term 1.
I20260619 08:55:25.030428 6342 leader_election.cc:302] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1277ef8abbfe48959c901b93ed0993d9, 1b616ee16391415fb425e9b6a9a57e47; no voters:
I20260619 08:55:25.029963 6324 master_runner.cc:186] Error getting master registration for 127.15.208.61:44041: OK, instance_id { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" instance_seqno: 1781859324826571 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:55:25.032819 6324 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:55:25.032883 6563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:55:25.033020 6563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [term 1 LEADER]: Becoming Leader. State: Replica: 1277ef8abbfe48959c901b93ed0993d9, State: Running, Role: LEADER
I20260619 08:55:25.033131 6563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } }
I20260619 08:55:25.033568 6582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1277ef8abbfe48959c901b93ed0993d9. Latest consensus state: current_term: 1 leader_uuid: "1277ef8abbfe48959c901b93ed0993d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } } }
I20260619 08:55:25.033792 6582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [sys.catalog]: This master's current role is: LEADER
W20260619 08:55:25.033996 6574 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:55:25.034055 6574 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:55:25.038658 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:25.030895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55914 (local address 127.15.208.62:43689)
0619 08:55:25.030930 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.030932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.031171 (+ 239us) server_negotiation.cc:407] Connection header received
0619 08:55:25.031344 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.031345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.031356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.031389 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.032154 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.032699 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.033860 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.034167 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.034210 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:25.035735 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:25.035736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:25.035736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:25.035742 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:25.037481 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.037846 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.037847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.037848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.037927 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.038138 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.038139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.038139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.038249 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:25.038353 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:25.038480 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 08:55:25.038566 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:55:25.046646 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:25.030763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.15.208.60:45397)
0619 08:55:25.030802 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.030815 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.038982 (+ 8167us) server_negotiation.cc:407] Connection header received
0619 08:55:25.039079 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.039081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.039092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.039122 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.039758 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.040195 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.041140 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.041448 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.041496 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:25.043397 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:25.043399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:25.043399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:25.043406 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:25.045119 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.045708 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.045709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.045709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.045768 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.045991 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.045992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.045992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.046128 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:25.046210 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:25.046431 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 08:55:25.046509 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:55:25.048866 6585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:55:25.049089 6585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:55:25.051717 6471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:25.031180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43152 (local address 127.15.208.61:44041)
0619 08:55:25.031222 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.031224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.040872 (+ 9648us) server_negotiation.cc:407] Connection header received
0619 08:55:25.040968 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.040970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.040982 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.041043 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.042043 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.042799 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.044072 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.044640 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.044689 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:25.046669 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:25.046670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:25.046671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:25.046690 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:25.048602 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.049070 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.049071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.049071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.049168 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.049422 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.049423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.049423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.049574 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:25.049662 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:25.051514 (+ 1852us) server_negotiation.cc:299] Negotiation successful
0619 08:55:25.051603 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:55:25.053678 6517 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [term 1 FOLLOWER]: Refusing update from remote peer 1277ef8abbfe48959c901b93ed0993d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:55:25.054001 6582 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:55:25.055393 6549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1277ef8abbfe48959c901b93ed0993d9. Latest consensus state: current_term: 1 leader_uuid: "1277ef8abbfe48959c901b93ed0993d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } } }
I20260619 08:55:25.055480 6549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:25.062194 6467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } attempt: 3
I20260619 08:55:25.065840 6582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1277ef8abbfe48959c901b93ed0993d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } } }
I20260619 08:55:25.065933 6582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:25.066550 6549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1277ef8abbfe48959c901b93ed0993d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } } }
I20260619 08:55:25.066632 6549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47 [sys.catalog]: This master's current role is: FOLLOWER
Jun 19 08:55:25 dist-test-slave-p0h9 krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781859324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.208.61@KRBTEST.COM for kudu/127.15.208.60@KRBTEST.COM
I20260619 08:55:25.068265 6585 catalog_manager.cc:1347] Generated new cluster ID: 9e97c5786ae440cb80e788b313b24e69
I20260619 08:55:25.068329 6585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:55:25.069286 6586 mvcc.cc:205] Tried to move back new op lower bound from 7298495795417214976 to 7298495795336359936. Current Snapshot: MvccSnapshot[applied={T|T < 7298495795417214976 or (T in {7298495795417214976})}]
I20260619 08:55:25.069433 6587 mvcc.cc:205] Tried to move back new op lower bound from 7298495795417214976 to 7298495795336359936. Current Snapshot: MvccSnapshot[applied={T|T < 7298495795417214976 or (T in {7298495795417214976})}]
I20260619 08:55:25.070449 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:25.062480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.208.60:45397 (local address 127.0.0.1:60944)
0619 08:55:25.062565 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 08:55:25.062576 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:55:25.062621 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:55:25.062757 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:55:25.062758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:55:25.062954 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:55:25.063005 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:25.063007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.063824 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.063825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:25.064628 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:55:25.064630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.065021 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.065022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:55:25.065090 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:25.065592 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:55:25.065597 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:55:25.067225 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:55:25.069211 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:25.069212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:25.069213 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:25.069451 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:25.069762 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:55:25.069762 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:55:25.069763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:55:25.069836 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:55:25.070137 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:55:25.070138 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:55:25.070181 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 08:55:25.070247 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 08:55:25.070356 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 08:55:25.071180 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:25.062543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60944 (local address 127.15.208.60:45397)
0619 08:55:25.062581 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.062583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.062672 (+ 89us) server_negotiation.cc:407] Connection header received
0619 08:55:25.062694 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.062695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.062704 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.062752 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.063081 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.063734 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.064709 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:25.064942 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:25.065025 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:25.067331 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:55:25.067333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:55:25.067333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:55:25.067339 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:55:25.069142 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.069577 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.069578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.069579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.069652 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:55:25.069913 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:55:25.069913 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:55:25.069914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:55:25.070070 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:55:25.070132 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:25.071042 (+ 910us) server_negotiation.cc:299] Negotiation successful
0619 08:55:25.071125 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:55:25.072541 6467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85: Bootstrap starting.
I20260619 08:55:25.073226 6467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85: Neither blocks nor log segments found. Creating new log.
I20260619 08:55:25.073496 6467 log.cc:824] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85: Log is configured to *not* fsync() on all Append() calls
I20260619 08:55:25.074280 6467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85: No bootstrap required, opened a new log
I20260619 08:55:25.076054 6467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } }
I20260619 08:55:25.076179 6467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93a3f8a91fb04b21badb68c7c3b4ad85, State: Initialized, Role: FOLLOWER
I20260619 08:55:25.076253 6467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } }
I20260619 08:55:25.076553 6467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:55:25.076522 6593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } } }
I20260619 08:55:25.076619 6440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:55:25.076678 6441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1277ef8abbfe48959c901b93ed0993d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85"
I20260619 08:55:25.077495 6440 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [term 1 FOLLOWER]: Refusing update from remote peer 1277ef8abbfe48959c901b93ed0993d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:55:25.076614 6593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:25.078559 6441 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1277ef8abbfe48959c901b93ed0993d9 in term 1.
I20260619 08:55:25.078930 6563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:55:25.080283 6593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1277ef8abbfe48959c901b93ed0993d9. Latest consensus state: current_term: 1 leader_uuid: "1277ef8abbfe48959c901b93ed0993d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } } }
I20260619 08:55:25.080364 6593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:25.080430 6607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1277ef8abbfe48959c901b93ed0993d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } } }
I20260619 08:55:25.080497 6607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:55:25.082855 6604 catalog_manager.cc:1259] Loaded cluster ID: 9e97c5786ae440cb80e788b313b24e69
I20260619 08:55:25.082926 6604 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85: loading cluster ID for follower catalog manager: success
W20260619 08:55:25.083015 6604 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260619 08:55:25.083060 6604 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:55:25.093127 6563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1277ef8abbfe48959c901b93ed0993d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1277ef8abbfe48959c901b93ed0993d9" member_type: VOTER last_known_addr { host: "127.15.208.62" port: 43689 } } peers { permanent_uuid: "93a3f8a91fb04b21badb68c7c3b4ad85" member_type: VOTER last_known_addr { host: "127.15.208.61" port: 44041 } } peers { permanent_uuid: "1b616ee16391415fb425e9b6a9a57e47" member_type: VOTER last_known_addr { host: "127.15.208.60" port: 45397 } } }
I20260619 08:55:25.093256 6563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:55:25.095634 6585 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:55:25.096386 6585 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:55:25.102435 6585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1277ef8abbfe48959c901b93ed0993d9: Generated new TSK 0
I20260619 08:55:25.102617 6585 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:55:25.130360 6547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.128594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.15.208.60:45397)
0619 08:55:25.128641 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.128644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.129681 (+ 1037us) server_negotiation.cc:407] Connection header received
0619 08:55:25.129787 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.129788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.129798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.129830 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.130285 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60954: BlockingRecv error: recv got EOF from 127.0.0.1:60954 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 08:55:25.131129 6395 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.129070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55924 (local address 127.15.208.62:43689)
0619 08:55:25.129122 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.129124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.130610 (+ 1486us) server_negotiation.cc:407] Connection header received
0619 08:55:25.130711 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.130712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.130723 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.130753 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.131053 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55924: BlockingRecv error: recv got EOF from 127.0.0.1:55924 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260619 08:55:25.131845 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.129417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.15.208.61:44041)
0619 08:55:25.129456 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.129458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.131352 (+ 1894us) server_negotiation.cc:407] Connection header received
0619 08:55:25.131449 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.131451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.131461 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.131491 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.131785 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43160: BlockingRecv error: recv got EOF from 127.0.0.1:43160 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:55:25.134163 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.133429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.15.208.61:44041)
0619 08:55:25.133475 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.133477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.133644 (+ 167us) server_negotiation.cc:407] Connection header received
0619 08:55:25.133746 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.133747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.133764 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.133792 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.134118 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43164: BlockingRecv error: recv got EOF from 127.0.0.1:43164 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 08:55:25.137349 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.136474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.15.208.61:44041)
0619 08:55:25.136509 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.136510 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.136842 (+ 332us) server_negotiation.cc:407] Connection header received
0619 08:55:25.136937 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.136938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.136948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.136976 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.137304 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43170: BlockingRecv error: recv got EOF from 127.0.0.1:43170 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:55:25.141295 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.140519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43184 (local address 127.15.208.61:44041)
0619 08:55:25.140550 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.140551 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.140786 (+ 235us) server_negotiation.cc:407] Connection header received
0619 08:55:25.140881 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.140882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.140891 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.140920 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.141251 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43184: BlockingRecv error: recv got EOF from 127.0.0.1:43184 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 08:55:25.147459 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.146497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.15.208.61:44041)
0619 08:55:25.146534 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.146535 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.146757 (+ 222us) server_negotiation.cc:407] Connection header received
0619 08:55:25.146858 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.146859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.146869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.146902 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.147409 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43198: BlockingRecv error: recv got EOF from 127.0.0.1:43198 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 08:55:25.157665 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.156789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.15.208.61:44041)
0619 08:55:25.156830 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.156832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.157105 (+ 273us) server_negotiation.cc:407] Connection header received
0619 08:55:25.157210 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.157211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.157222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.157252 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.157610 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43202: BlockingRecv error: recv got EOF from 127.0.0.1:43202 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 08:55:25.176066 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.175116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.15.208.61:44041)
0619 08:55:25.175162 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.175164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.175438 (+ 274us) server_negotiation.cc:407] Connection header received
0619 08:55:25.175549 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.175550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.175563 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.175595 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.176005 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43206: BlockingRecv error: recv got EOF from 127.0.0.1:43206 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 08:55:25.211000 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.209779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43214 (local address 127.15.208.61:44041)
0619 08:55:25.209832 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.209834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.210119 (+ 285us) server_negotiation.cc:407] Connection header received
0619 08:55:25.210235 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.210236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.210248 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.210281 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.210940 (+ 659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43214: BlockingRecv error: recv got EOF from 127.0.0.1:43214 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260619 08:55:25.277441 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.276684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43226 (local address 127.15.208.61:44041)
0619 08:55:25.276767 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.276769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.276864 (+ 95us) server_negotiation.cc:407] Connection header received
0619 08:55:25.276925 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.276926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.276936 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.276991 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.277369 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43226: BlockingRecv error: recv got EOF from 127.0.0.1:43226 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 08:55:25.407341 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.406584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.15.208.61:44041)
0619 08:55:25.406657 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.406659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.406800 (+ 141us) server_negotiation.cc:407] Connection header received
0619 08:55:25.406849 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.406850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.406860 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.406914 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.407249 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43230: BlockingRecv error: recv got EOF from 127.0.0.1:43230 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 08:55:25.665470 6471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:55:25.664801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.15.208.61:44041)
0619 08:55:25.664877 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:25.664880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:25.664890 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:55:25.664941 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:25.664942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:25.664957 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:25.665032 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:55:25.665408 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43238: BlockingRecv error: recv got EOF from 127.0.0.1:43238 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:55:26.012856 6560 catalog_manager.cc:1259] Loaded cluster ID: 9e97c5786ae440cb80e788b313b24e69
I20260619 08:55:26.012926 6560 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47: loading cluster ID for follower catalog manager: success
I20260619 08:55:26.013756 6560 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47: acquiring CA information for follower catalog manager: success
I20260619 08:55:26.014067 6560 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b616ee16391415fb425e9b6a9a57e47: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:55:26.084160 6604 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85: acquiring CA information for follower catalog manager: success
I20260619 08:55:26.084563 6604 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 93a3f8a91fb04b21badb68c7c3b4ad85: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:55:26.181468 6675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:26.178814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.15.208.61:44041)
0619 08:55:26.178930 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:26.178932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:26.178943 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:55:26.178994 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:26.178995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:26.179043 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:26.179131 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:55:26.179349 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:26.179859 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:26.180627 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:26.180855 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:26.180913 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:26.181073 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:55:26.181186 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:55:26.181245 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:26.181309 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 08:55:26.181327 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
I20260619 08:55:26.185588 6681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:26.183356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60958 (local address 127.15.208.60:45397)
0619 08:55:26.183472 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:26.183474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:26.183507 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:55:26.183550 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:26.183551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:26.183587 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:26.183651 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:55:26.183746 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:26.184231 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:26.184955 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:26.185140 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:26.185205 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:26.185322 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:55:26.185423 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:55:26.185478 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:26.185509 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 08:55:26.185526 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":71,"threads_started":1}
I20260619 08:55:26.189920 6681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:26.187411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.15.208.60:45397)
0619 08:55:26.187447 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:26.187449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:26.187484 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:55:26.187536 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:26.187537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:26.187547 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:26.187591 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:55:26.187746 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:26.188325 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:26.189252 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:26.189430 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:26.189497 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:26.189696 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:55:26.189724 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:55:26.189774 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:26.189829 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 08:55:26.189848 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:55:26.190454 6675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:26.187355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43252 (local address 127.15.208.61:44041)
0619 08:55:26.187457 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:26.187459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:26.187578 (+ 119us) server_negotiation.cc:407] Connection header received
0619 08:55:26.187780 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:26.187781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:26.187795 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:26.187840 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:55:26.188026 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:26.188630 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:26.189631 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:26.189908 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:26.189970 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:26.190241 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:55:26.190272 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:55:26.190319 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:26.190365 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 08:55:26.190384 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 08:55:26.191027 6688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:55:26.187315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55926 (local address 127.15.208.62:43689)
0619 08:55:26.187416 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 08:55:26.187418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:55:26.187465 (+ 47us) server_negotiation.cc:407] Connection header received
0619 08:55:26.187512 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:55:26.187512 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:55:26.187567 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:55:26.187645 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:55:26.188419 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:26.189173 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:26.190137 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:55:26.190435 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:55:26.190503 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:55:26.190667 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:55:26.190772 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:55:26.190865 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 08:55:26.190921 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 08:55:26.190942 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":56,"threads_started":1}
I20260619 08:55:26.196105 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 6324
I20260619 08:55:26.201472 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 6399
I20260619 08:55:26.206444 16192 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLqpuKl/build/release/bin/kudu with pid 6474
2026-06-19T08:55:26Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2837 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5633 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (268462 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST